Currently logcheck focuses on /var/log/syslog and /var/log/auth.log but it would be nice to have separate filtering for other types of logs that normally don't get merged into syslog or the journal. One of those types of logs is apt upgrade logs. When apt itself is invoked, it sends terminal output (including \r but not colours) to the apt term.log file. If unattended-upgrades is being run then the same output and also output of the apt hooks goes to the additional log file unattended-upgrades-dpkg.log (which also contains \r but not colours). The unattended-upgrades code may also send a mail with that log output but without any \r or colors. The unattended-upgrades wrapper around apt print various uninteresting messages. The cron-apt/aptitude-robot alternatives probably also do the same. apt itself prints a lot of messages that aren't interesting. The apt hooks shipped by various packages print various uninteresting messages. The maintainer scripts shipped by various packages print various uninteresting messages. I'm currently using the attached hacky script with lots of regexes to implement apt upgrade log filtering. It seems to me that a better way to do this would be to integrate separate apt filtering into logcheck and then integrate into each package (including apt) logcheck ignore configs containing the regexes that represent uninteresting messages. There could be an apt hook to use logcheck to filter the apt term.log to an apt term-interesting.log and an unattended-upgrades hook to filter unattended-upgrades-dpkg.log to a corresponding file and an option/hook to filter unattended-upgrades mails. The same could be done for cron-apt/aptitude-robot.
Hi Paul If my memory is correct. I should know since I wrote the cron-apt software but it was quite some time ago. ... Checking the code now to refresh my memory... Emails are sent by default only when apt fails. All output is logged to the cron-apt log file. Logs are sent to syslog on "upgrade". By default upgrade is not enabled. I'm not sure this helps in any way, but I thought I could at least respond with what I know. Cheers // Ola
Hi Paul If my memory is correct. I should know since I wrote the cron-apt software but it was quite some time ago. ... Checking the code now to refresh my memory... Emails are sent by default only when apt fails. All output is logged to the cron-apt log file. Logs are sent to syslog on "upgrade". By default upgrade is not enabled. I'm not sure this helps in any way, but I thought I could at least respond with what I know. Cheers // Ola
Thanks for that info. If this feature gets implemented in logcheck, there could be cron-apt options to apply it to either syslog or mails for each of the types of output cron-apt sends. An option to send mail/syslog for success too would be good for users of this logcheck feature and possibly also users who do processing of the results either in a mail client plugin or in a syslog database and search system like Elasticsearch or Loki.
Hi Paul
The current possibilities in cron-apt are:
SYSLOGON="upgrade"
When to log the cron-apt results to syslog.
Value:
error (syslog on error runs)
upgrade (when packages is upgraded)
changes (syslog when change in output from an action)
output (syslog when output is generated)
always (always syslog)
never (never syslog)
(else never syslog)
MAILON="error"
When to send email about the cron-apt results.
Value:
error (send mail on error runs)
upgrade (when packages are upgraded)
changes (mail when change in output from an action)
output (send mail when output is generated)
always (always send mail)
never (never send mail)
(else never send mail)
I guess "output" is what you are after.
// Ola
Hi Paul, Paul Wise wrote: aptitude-robot does this in https://github.com/elmar/aptitude-robot/blob/master/reporting-helpers which is sourced by the actual hooks in https://github.com/elmar/aptitude-robot/blob/master/mail-log-on-error and https://github.com/elmar/aptitude-robot/blob/master/xymon-report But it does this on its own log file which captures the whole terminal output of dpkg/apt/aptitude. Regards, Axel
Interesting. This detects instances of a few "warning" and "error" types of strings. This is different to the approach I was going for where logcheck would hide "uninteresting" strings and highlight everything else. Perhaps logcheck could add these different levels of interest in messages too, so you could say only show me errors or show me warnings and errors or show me everything "interesting".
Here is an updated version of the script.
I'm sorry, but i read the bug and i am not entirely sure i understood what the request for logcheck is! but if you want to use logcheck to scan apt logs this is already possible. You just need to add the name of the log to a file in the logfiles.d directory. Your script using sed could be then be replaced by a list of regexps and they would be used by logcheck in its usual run, and matches reported in the usual way. (I do this locally on other log files and it works. One limitation is that logcheck cannot report which line came from which log and i wouldnt see that changing) If you want to run logcheck totally separately from the periodic check, with different config, so that the reported lines are not mixed up with the normal log entries (eg in an apt hook), this is also possible - you will need to invoke logcheck as the logcheck user and use the relevent command line options. (the autopkgtests do this). .... But if this is really about having apt/unattended-uogrades report errors better it would be better to do that change directly in those tools i would suggest I would also echo the recommendion for using cron-apt which can already send an email (or write a log entry for logcheck to report) if anything went wrong, (or always - apart from cron-apt not yet having a systemd timer it is quite easy to configure and works very well). it seems better to use that existing functionality than to insert logcheck. but perhaps i didnt understand the request.
Thanks for the info and thoughts. The idea would do something like your second suggestion; run logcheck on apt logs separately, but within Debian instead of just on my system. Perhaps we could also distribute the ignore regexes across packages like logcheck itself does. Since not everyone is going to want this, probably this would be in a separate apt logcheck package. Since apt term.log collects all apt output, but doesn't collect output from apt hooks (which is often useful), unattended-upgrades-dpkg.log collects only automated apt output, but *does* collect output from apt hooks, this idea is going to require some changes to one or either of those packages so that users don't get duplicate entries. The apt/u-u tools already report errors fine, this isn't about that, but about reporting anomalous messages that aren't apt errors, like notices or warnings from maintainer scripts or apt hooks etc. The cron-apt error check is likewise not useful for this idea. PS: do you know if logcheck supports printing nearby ignored lines for context for each non-ignored log message that was printed? That would be essential for this apt log filtering because the non-ignored messages are usually produced by package maintainer scripts and the ignored nearby lines contain the name of the relevant packages.
(i wonder how well this distributed model works - i dont think most packages have understood the violations layer, and rules do not seem to be kept updated (also true of logcheck-database). it also means logcheck-database is less useful for other consumers, and creates cruft for people that do not use logcheck or who want to write all the rules themselves) it doesnt out of the box - yet: but adding support for replacing grep wih faster tools like rg or ag is already on my list for bookworm and the implementation-in-progress allows other options such as -C etc to be passed through. I assume grep does something sensible when outputting context with -f but there would be some subtleties with how it interacts with the violations and violations-ignore layers and sorting/collating of rules.... simplifying that mess is also on the list.