Towards the purpose of making much tigher matches easier, I'd like to
propose implementing the ability to use macros in the pattern files. The
patch below is one way this might be done. Create $CONFDIR/macros, and
have lots of lines of the form "name=stuff" and $name in the rule files
will be substituted.
A sample set of macros might be:
PID='[[:digit:]]+'
IPV4ADDR=[:digit:]+\.[:digit:]+\.[:digit:]+\.[:digit:]+
IPV6ADDR='[[:xdigit:]]{1,4}(:[[:xdigit:]]{1,4}){7}'
IPADDR="($IPV4ADDR|$IPV6ADDR)"
DATE='(Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec) [ 123][0-9] [ 12][0-9]:[0-5][0-9]:[0-5][0-9]'
HOSTNAME='[_-[:alnum:]][._-[:alnum:]]+'
BEGIN="^$DATE $HOSTNAME"
I'm sure there are other ones people might wish to add, and these can be
made tighter. It is rather unlikely to be getting reports from single
digit PIDs (and if there are, these likely should be getting reportted).
The IP address representations can be confined to tighter ranges too.
--- logcheck.part1 Wed Dec 25 23:12:03 2002
+++ logcheck Wed Dec 25 23:30:46 2002
@@ -90,24 +90,46 @@
dir=$1
cleaned=$2
+ cleantmp=$cleaned/#cleantmp#
+
if [ -d $dir ]; then
- debug "Cleaning $dir to directory"
- $MKDIR $cleaned
+ debug "Cleaning $dir to directory $cleaned"
+ # This might actually an indicator of a severe cracking attempt...
+ if ! $MKDIR $cleaned; then
+ error $ECHO "cleanrules: Failed to create directory $cleaned"
+ fi
+
# This should be replace by run-parts -l
for rulefile in $(ls -1B --ignore="#*#" $dir/); do
case "$rulefile" in
*.dpkg*) ;;
*)
+ debug "Cleaning $(basename $dir) to directory: $rulefile"
+
if [ -h ${dir}/${rulefile} ]; then
debug "$dir/$rulefile is a symlink - In a future release this file will be silently ignored"
$ECHO "$dir/$rulefile is a symlink - In a future release this file will be silently ignored" >&2
- $GREP -v '^\s*$|^#' $dir/$rulefile >> $cleaned/$rulefile
- elif [ -f ${dir}/${rulefile} ]; then
- debug "Cleaning $(basename $dir) to directory: $rulefile"
- $GREP -v '^\s*$|^#' $dir/$rulefile >> $cleaned/$rulefile
fi
+
+ r0=`$ECHO 'cat << __EOF__' > $cleantmp`
+ r1=`$GREP -v '^\s*$|^#' $dir/$rulefile >> $cleantmp`
+ r2=`$ECHO '__EOF__' >> $cleantmp`
+ # We deliberately test this now *before* running through sh
+ if [ $r0 -ne 0 -o $r2 -ne 0 -o $r1 -gt 2 ]; then
+ error "cleanrules: $GREP failed, full FS?"
+ fi
+ # It is ordained that the filter configs are not executable
+ sed -e s/\`/\\\`/g -e s/\$\(/\\\$\(/g < $cleantmp | \
+cat $CONFDIR/macros /dev/stdin | /bin/sh > $cleaned/$rulefile
+
+ # Hmm, filesystem full perhaps?
+ if [ $? -ne 0 ]; then
+ error "cleanrules: Failed to clean rules, full FS?"
+ fi
+ ;;
esac
done
+ rm $cleantmp
elif [ -f $dir ]; then
error "cleanrules: '${dir}' is a file, not a diectory" >&2
elif [ -z $dir ]; then
-8<--8<-
Note that while I've currently got 1.1.1-13.1 installed, the provided patch is against 1.1.9.8.
A version of the patch against 1.2.0:
Elliott, I like this idea, but I'm thinking that it should be done though another script (e.g. dh_installlogcheck - which I plan to start writing soon) instead of in Logcheck it's self. Also I think that the new macros should take the same format as the grep macros (i.e. [:ipv4addr:], [:ipv6addr:] [:date:] and [:hostname:]), if these could be added to grep it would be even better.
It should be kept with the rule cleaning though. dh_installlogcheck sounds like the wrong place, since part of the point is to make it easier to adjust the rules there needs to be a place to run the files through the macro system other than at install time. Also a sysadmin might want to change BEGIN to be specific to their host (be able to detect that somebody from outside is dumping their syslog onto you). Combined with an appropriate END macro you might be a lot closer to being able to support things like #159223. This strikes me as a *bad* idea. The [:<type>:] classes are exclusively single characters; the macro support I'm suggesting is for multicharacter strings which is very different. These are also definitly non-general purpose so there is little hope support would be added to grep. Ideally the rule files could define their own macros, but I couldn't think of a way to do that without opening an even larger can of worms.
[patch snipped] Elliott, The old behavior was there for a reason, a rulefile that has a line with a single space will exclude all log messages with a space in it. I've looked at your patch and it does not function. Also I have a feeling that it would break the following pattern: kernel: usb-uhci.c: $Revision: 1.275 $ time 22:14:52 Sep 23 2002 Macro substitution must not have any effect on the contents of the rulefiles other than carring out the macro substitution and your implementation does not fulfil this. In summary when you send a patch to the BTS could you make sure that it a) works as advertised. b) fits the coding style. And most importantly c) does not break current functionality. Thanks,
Yes, it is unlikely somebody wants to skip all messages (or note all messages), but it is a valid thing to do. A more interesting use is to grab messages with two spaces in a row. I'd hoped you would consider it although I fully expected you to add it back in. Yes, it would swallow "$Revision" for substitution; however, I note that none of the sample files contain $ anywhere, and what is likely the current common case (telling egrep where the EOL should be) is not affected. Doing macro substitution will impact rules that do not use it no matter what, because those will have to avoid the magic strings used for substitution. My patch uses $, it was simple, doesn't mess up any of the provided rules, and /likely/ avoids most rules written by others; it is a reasonable choice, but like any there are sure to be weaknesses under some conditions. It did, I warned you I'd done minimal verification and noted the one difference that I'd lost fighting with sed's regex. Nah I thought I'd convert the entire thing to 1TBS while I was at it... I didn't expect a perfect match of your coding style, just to merely not stick out too badly. Modify as you feel is needed. I expected you'd add that back in if you thought it was still needed. This time I've checked to make sure it works the way I'd planned it to, please see the note at the bottom about space lines...
there have been several requests for macros. could you update your patch against a current logcheck version (sarge 1.2.28, sid 1.2.29, cvs 1.2.30) are quite similar. your ideas for checking the return of any command in logcheck has already been implemented inbetween. it would be great if your patches focusses on macro only. please no whitespace cleanup. thanks a lot + hope to read you soon maks ps if you have any question, don't hesitate to ask.
I'd be very interested in seeing this patch applied to logcheck. Have the maintainers made a final decision on this one, yet? Thanks, - Marc
well no clean patch surfaced of that bug thread yet, so the decision is easy. it's post sarge. working contributions are always welcome of course. :-)
patch against git master I had to changed my rsyslog format recently and logcheck is near to unusable if I need to change/duplicate/transform all the rules
maintainer?