Tony Finch – blog

So this weekend I was playing around with my log summarizer, which produces a line of output for each second with one character for each significant event coloured according to the machine on which it occurred. The output is all real-time, and quite mesmerizing to watch. Behind the cut is a static sample of the output.

All my email relays are logging via syslog to our admin box, which writes the output to a fifo. The summarizer can read the fifo directly, but that means only one person can view it at a time, and I can’t do anything else with the log data feed while it is running.

So I wrote a little multiplexing program which will reads from a pipe or a fifo, and listens on a socket (unix domain or internet). Clients can connect to the socket and the multiplexer feeds them all a copy of the data it reads from the input. If the multiplexer is listening on a network socket then telnet is a suitable client, but I wrote a simpler one for use with unix domain sockets.

So the result of this is that I can write:

logmux -f /var/log/ppsw_fifo -s /home/fanf2/logs/test_sock -c &

so whenever I connect to test_sock I get a copy of the amalgamated ppsw log feed.

loguse -s /home/fanf2/logs/test_sock | logfilter | logmux -s :2000

and I can peel a feed off, pipe it through my shiny summarizer, and provide the output as a service on port 2000. I can then just telnet to it and get a feed.

It seems to me that logmux and loguse might turn out to be quite useful in the future.

sample log summary output

A quick key to the below:

• < is a message coming in.
• > is a message going out, and - is used when the same copy is delivered to multiple recipients.
• Lower-case j is a message that has been accepted and which SpamAssassin thinks is spam;
• similarly v is used for viruses (which are usually deleted).
• O is for rejected relaying attempts,
• A is for attempted unauthorized use of our smarthost services,
• H is a rejection because of incorrect HELO domain,
• P is rejection because of SMTP protocol violation,
• D is rejection because of DNS blacklisting.
• S is rejection due to invalid sender address, and lower-case s is temporary failure to validate the sender.
• R is rejection because of invalid recipient adress, and the rare lower-case r is temporary failure to validate the recipient (which we are usually lenient about).

The first number on the right is the smoothed number of actions per second, the second number is the percentage of email that is rejected, and the third number is the percentage of accepted email that SpamAssassin thinks is junk.