This site is now 100% read-only, and retired.

Producing and viewing mailserver statistics with isoqlog

Posted by Steve on Sat 27 Aug 2005 at 12:41

If you wish to keep track of the mail sent and received by a machine then isoqlog is a nice solution allowing you to view your top mail senders and receivers. It works with exim, postfix, sendmail, and qmail.

Whilst it doesn't provide a full monitoring solution in the same way that munin, or cacti would it does allow you to keep track of mailserver performance and throughput in a readable fashion.

On many hosts keeping statistics on your mailserver, and performing webserver logfiles analysis is all that is required anyway.

For a look at what it offers please see the sample output. This shows you the kind of behaviour you can expect to see upon your hosts.

Output is broken down into three fields:

  • Sent
    • Keeping track of mails sent from your host.
  • Received
    • Keeping track of incoming mails addressed to your host.
  • Size
    • The size of mails sent or received.

The outputs is further broken down by domain name.

To install the software you can simply run:

apt-get install isoqlog

(Or if you wish "aptitude install isoqlog".)

Once installed you may be prompted by debconf to configure it. On my Debian Sarge installation this did not occur, but it did on the two Debian Sid hosts I tested it upon. Currently the same version will be installed on Sid, and Sarge, so I'm not sure where the difference comes from - and your result may vary.

To configure the software we need to setup several variables:

  • The logtype exim/sendmail/etc.
  • The input logfile to use.
  • The domain names to include in the output.
  • The system's hostname.
  • The directory to place output files in.

If prompted by debconf enter your choices. If you're not then you can setup the various configuration settings in the file /etc/isoqlog/isoqlog.conf manually.

The domain names you wish to monitor should instead be entered, one per line, in /etc/isoqlog/isoqlog.domains.

For example this sample shows the setup I use upon an exim4 mailserver:

logtype     = "exim"
logstore    = "/var/log/exim4/mainlog"
domainsfile = "/etc/isoqlog/isoqlog.domains"
outputdir   = "/var/www/isoqlog"
htmldir     = "/usr/share/isoqlog/htmltemp"
langfile    = "/usr/share/isoqlog/lang/english"
hostname    = "mystery"

maxsender   = 100
maxreceiver = 100
maxtotal    = 100

maxbyte     = 100

Here I've changed the "logstore" to point to exim4's logfile, and setup the output directory to /var/www/isoqlog - where my Apache installation can serve it from - with appropriate access control in place to avoid leaking private information.

The only other change I made was to set the langfile to /usr/share/isoqlog/lang/english - the default was to use English (with a capital "E") which didn't exist.

That is sufficient to setup the software.

By default there is a cronjob installed in /etc/cron.daily/isoqlog to generate the statistics once per day.

If you do not wish to wait, and wish to generate some immediately run the following command manually, as root:

/usr/bin/isoqlog

 

 


Re: Producing and viewing mailserver statistics with isoqlog
Posted by Serge (213.118.xx.xx) on Sat 27 Aug 2005 at 13:29
[ View Weblogs ]
If you want some nice graphs and use Postfix, you might consider using the postgraph package. If you use amavis filtering, you also have amavis-stats. They al use the RRD engine from mrtg to produce some nice graphs.

Yes, log file analysis can come very handy. E.g. when noticing a sudden boost of the graph in 1 hour, flattening all you other data makes you discovering you were hacked by a spammer.
Don't you hate cross site scripting?

D'oh


--

Serge van Ginderachter


[ Parent ]

Re: Producing and viewing mailserver statistics with isoqlog
Posted by Anonymous (82.119.xx.xx) on Sat 27 Aug 2005 at 15:04
Sometimes that boost means new virus is active :)
The package is called mailgraph, not postgraph. I also use couriergraph for monitoring pop3/imap connections.
Some time ago I tried isoqlog, but I had problems with many domains. I'm not sure if it was mandatory to have a list of all domains I accept mail to, or it was just for having the stats split by domain...

[ Parent ]

Re: Producing and viewing mailserver statistics with isoqlog
Posted by Serge (213.118.xx.xx) on Sat 27 Aug 2005 at 21:48
[ View Weblogs ]
Sometimes that boost means new virus is active :)
Sometimes maybe, but here the difference was toooooo big. I'm talkign about 100 times the average traffic I get, within one continuous hour.
I found enough live evidence of an active intrusion. wget http://www.fullteam.net/... in teh apache bash_history (d'oh, yes) and connections from backdoors to this server: http://201.29.248.103/ (think XSS). You get the feeling? It got late yesterday :)


--

Serge van Ginderachter


[ Parent ]

Re: Producing and viewing mailserver statistics with isoqlog
Posted by Steve (82.41.xx.xx) on Sat 27 Aug 2005 at 23:51
[ View Weblogs ]

I firewall outgoing HTTP connections from important hosts for precisely this reason...

It's hard to prevent XSS attacks though if you're running software you've not audited yourself - or if you've got customers running random CGI/PHP code.

Even mod_security won't help if you've not got appropriate rules setup.

If the bug which was exploited isn't obviously public it could be worth reporting to the debian-isp / debian-security mailing lists.

Steve
-- Steve.org.uk

[ Parent ]

Re: Producing and viewing mailserver statistics with isoqlog
Posted by Anonymous (212.18.xx.xx) on Sat 27 Aug 2005 at 15:25
eximstats?

Comes with exim...

[ Parent ]

Re: Producing and viewing mailserver statistics with isoqlog
Posted by Steve (82.41.xx.xx) on Sat 27 Aug 2005 at 17:37
[ View Weblogs ]

Good point, but if you're not using exim?

Steve
-- Steve.org.uk

[ Parent ]

Re: Producing and viewing mailserver statistics with isoqlog
Posted by zorg (82.230.xx.xx) on Mon 29 Aug 2005 at 09:10
hello
for myself, I would prefer awstats
here is an example
http://awstats.sourceforge.net/awstats.mail.html


[ Parent ]

Re: Producing and viewing mailserver statistics with isoqlog
Posted by hardik (61.95.xx.xx) on Mon 29 Aug 2005 at 09:33
Very nice utility i like it very much, I am running on my
production server which is running Qmail with qmail-ldap patch. But can't i have query option so i can query for perticular timebeing....i.e. Report beetween 29-08-05 to 05-09-05 fort user test. And more, Report of Top 100 Size doesn't show that user send or recieve this mail with given size.

Tips,
I am running svlogd and runit in my m/c. So User who are running svlogd can achieve same logtype as multilog supported with -t option.


With Cheers,
Hardik Dalwadi.

[ Parent ]