Splitting Syslogs by Facility

Logs are a wonderful thing. If done correctly they point out the source of all errors, show you what’s running slow and contain useful information on how your system is running. At every place I’ve ever worked they’ve been busy, full of odd one offs and too often overlooked.

I’m going to be doing a fair bit of log processing next week so expect lots of little toolchain scripts like syslog-splitter.pl to be checked in to git and mentioned here.

syslog-splitter takes a logfile as an argument and breaks the logfiles in to many smaller units, one file per facility (which contains all the lines for that facility from the logfile), to make it easier to process. I seem to invoke it followed by wc -l out/* | sort -nr when on new machines to work out where I need to invest some time. Over the next week or so I’ll come back to the topic and show how I’m reducing the noise to help me find the important lines.