#609940 Support RSYSLOG_FileFormat (RFC 3339 timestamps)

#609940#5
Date:
2011-01-14 02:36:28 UTC
From:
To:
In rsyslog.conf, there is

    # Use traditional timestamp format.
    # To enable high precision timestamps, comment out the following line.
    #
    $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

When it's commented out, I get lines like these:

    2011-01-14T13:02:48.036142+11:00 venus dhclient: DHCPREQUEST of 192.168.155.148 on managed to 192.168.155.20 port 67
    2011-01-14T13:02:48.052813+11:00 venus dhclient: DHCPACK of 192.168.155.148 from 192.168.155.20
    2011-01-14T13:02:48.134247+11:00 venus dhclient: bound to 192.168.155.148 -- renewal in 1525 seconds.
    2011-01-14T13:03:47.100359+11:00 rifle dhclient: DHCPREQUEST of 192.168.155.160 on managed to 192.168.155.20 port 67
    2011-01-14T13:03:47.127127+11:00 rifle dhclient: DHCPACK of 192.168.155.160 from 192.168.155.20
    2011-01-14T13:03:47.177701+11:00 rifle dhclient: bound to 192.168.155.160 -- renewal in 1435 seconds.
    2011-01-14T13:05:01.100265+11:00 alamo dhclient: DHCPREQUEST of 192.168.155.162 on managed to 192.168.155.20 port 67
    2011-01-14T13:05:01.188194+11:00 alamo dhclient: DHCPACK of 192.168.155.162 from 192.168.155.20
    2011-01-14T13:05:01.226055+11:00 alamo dhclient: bound to 192.168.155.162 -- renewal in 1203 seconds.

That's OK for logcheck -- its regexps are stored in conffiles, so I
can just change the prefix for each line to match the new RFC 3339
format.

For syslog-summary, the regexps are embedded within the script (in
/usr/bin), so I can't do so.  It would be nice if I could.

PS: note that the nanosecond field is absent for lines received by
rsyslog from a traditional syslogd, or from an rsyslogd without

    $ActionForwardDefaultTemplate RSYSLOG_ForwardFormat

In other words, the solution to this ticket must also recognize lines like

    2011-01-14T13:34:40+11:00 groan rsyslogd: [origin software="rsyslogd" swVersion="4.2.0" x-pid="22184" x-info="http://www.rsyslog.com"] (re)start

#609940#10
Date:
2015-01-15 03:32:32 UTC
From:
To:
I set up another logserv today, so I found this bug again.
I've been using the monkey patch below,
but it looks like I never put it on this ticket.

    aptitude install -yq syslog-summary
    cp -p /usr/bin/syslog-summary /etc/syslog-summary.py
    dpkg-divert --rename /usr/bin/syslog-summary
    ln -s /etc/syslog-summary.py /usr/bin/syslog-summary
    etckeeper commit 'Make syslog-summary(1) a conffile, so it can be fixed.'
    sed -i /etc/syslog-summary.py \
        -e '1c#!/usr/bin/python' \
        -e 's/^datepats =.*/& re.compile(r"^[0-9T.:+-]{25,32} "),/'
    etckeeper commit 'Teach syslog-summary about granular timestamps (and python2.6).'
    echo SYSLOGSUMMARY=1 >>/etc/logcheck/logcheck.conf
    etckeeper commit 'Have logcheck use syslog-summary.'

I extended it to recognize and remove the timestamps that appear in
kern.log these days, see diff below.

    diff --git a/usr/bin/syslog-summary.distrib b/etc/syslog-summary.py
    index 17ad38b..a9a01dc 100755
    --- a/usr/bin/syslog-summary.distrib
    +++ b/etc/syslog-summary.py
    @@ -1,4 +1,4 @@
    -#!/usr/bin/env python2.5
    +#!/usr/bin/python
     # -*- coding: utf-8 -*-

     # Copyright © 2008-2009, David Paleino <d.paleino@gmail.com>
    @@ -43,11 +43,13 @@ from hashlib import sha1
     from optparse import OptionParser

     datepats = [
    +        re.compile(r"^[0-9T.:+-]{25,32} "), # RFC3339 timestamps
            re.compile(r"^(Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec) [ 0-9][0-9] [ 0-9][0-9]:[0-9][0-9]:[0-9][0-9] "),
            re.compile(r"^(Mon|Tue|Wed|Thu|Fri|Sat|Sun) (Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec) [ 0-9][0-9][0-9][0-9]:[0-9][0-9] "),
            re.compile(r"^(Mon|Tue|Wed|Thu|Fri|Sat|Sun) (Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec) [ 0-9][0-9][0-9][0-9]:[0-9][0-9]:[0-9][0-9] "),
     ]
     pidpat = re.compile(r"^([^ ]* [^ ]*)\[[0-9][0-9]*\]: ")
    +pidpat = re.compile(r"^([^ ]* [^ ]*)(\[[0-9]+\]:|: \[[ 0-9.]+\]) ") # match pids OR printk timestamps
     repeatpat = re.compile(r"^[^ ]* last message repeated (\d+) times$")

     ignore_pats = []

#609940#15
Date:
2020-05-10 03:01:07 UTC
From:
To:
Dear submitter,

as the package syslog-summary has just been removed from the Debian archive
unstable we hereby close the associated bug reports.  We are sorry
that we couldn't deal with your issue properly.

For details on the removal, please see https://bugs.debian.org/959993

The version of this package that was in Debian prior to this removal
can still be found using http://snapshot.debian.org/.

Please note that the changes have been done on the master archive and
will not propagate to any mirrors until the next dinstall run at the
earliest.

This message was generated automatically; if you believe that there is
a problem with it please contact the archive administrators by mailing
ftpmaster@ftp-master.debian.org.

Debian distribution maintenance software
pp.
Scott Kitterman (the ftpmaster behind the curtain)