logo

 

This site is now 100% read-only, and retired.

SNMP and MRTG on Sarge quick start

Posted by cswd on Mon 6 Mar 2006 at 09:40

Tags: , ,

There appear to be no uncomplicated introductions to the subject of traffic monitoring on the internet anywhere. So here is one. The objective is to get traffic graphs for the primary interface on your server, workstation or firewall quickly and efficiently.

The system is fairly simple and consists of these parts:

  1. The SNMP server. This allows access via the SNMP protocol to the system's network interface statistics and other data.
  2. The MRTG (Multi router traffic grapher). This is a large Perl script which polls the SNMP server and accumulates information about network usage. This runs periodically from cron and generates graphs at defined intervals

Please note that you can collect and graph anything with these - they are not limited to network statistics. Not only that, you can manage many aspects of your server with SNMP.

Please make sure you have apache or apache2 installed for this to work.

SNMP server configuration

Firstly, you need an SNMP server to provide network interface statstics on demand:

# apt-get install snmpd

You need to edit the configuration for this as it does not allow any connections by default. With your favourite editor, edit:

/etc/snmp/snmpd.conf

Comment out the following (prefix with #):

com2sec paranoid default public

Insert the following underneath the commented out section:

com2sec readonly default public

That gives anyone with access to the SNMP server read-only access to the public community. This is the one that contains the interface statistics.

To apply the changes, restart snmpd:

/etc/init.d/snmpd restart

Make sure you firewall off any SNMP related ports so that you don't get any unwanted visitors (check netstat and /etc/services for port information).

Installation of MRTG

MRTG is the main collection and graphing component of the traffic monitoring solution I am presenting here. Firstly, install MRTG:

# apt-get install mrtg

You can manually or automatically generate the configuration file for mrtg. I would recommend doing it automatically as it is a lot easier. Issue the following command:

# cfgmaker --global 'WorkDir: /var/www/mrtg' \
    --output /etc/mrtg.cfg public@127.0.0.1

This will generate the configuration file. You then need to make an index file which contains a list of all of your interfaces. Issue the following command:

# indexmaker /etc/mrtg.cfg --columns=1 \
    --output /var/www/mrtg/index.html

You will now need to execute mrtg manually 3 times to create the required database files. Issue the following command 3 times sequentially. On the third run, you should see no errors being reported:

# mrtg

This is executed every 5 minutes by cron. The cron job was added by dpkg for you so you do not have to configure it.

Conclusion

Finally, inspect your results! You will not see any reasonable graphs for quite some time so sit back end relax for a bit!

Browse to http://your-server-name/mrtg/

You can see my own freshly configured uninteresting mrtg statistics here: http://www.cswd.co.uk/mrtg/

Further reading

 

 

 

#
Re: SNMP and MRTG on Sarge quick start
Posted by Anonymous (84.194.xx.xx) on Mon 6 Mar 2006 at 10:20
I'm using cacti to view the details of the statistics. It creates "smarter" graphics and it has a web-based config to add other snmp sources (hosts).

apt-get install cacti

Cacti: The Complete RRDTool-based Graphing Solution
(while writing this comment, cacti.net was offline ...)


Fred
Linox.BE

[ Parent ]

#
Re: SNMP and MRTG on Sarge quick start
Posted by cswd (62.255.xx.xx) on Mon 6 Mar 2006 at 11:01
Cacti is fine but it has caused performance problems on non-dedicated servers (like my web server VM). In my circumstance, mrtg has a minimal footprint and is quite easy to integrate with the web infrastructure.

[ Parent ]

#
Re: SNMP and MRTG on Sarge quick start
Posted by stephanhuiser (82.93.xx.xx) on Mon 6 Mar 2006 at 10:47
After putting /usr/bin/mrtg in my crontab, I receive the following messages from my cron daemon:

ERROR: It looks as if you are running two copies of mrtg in parallel on
the same config file. There is a lockfile (/var/lock/mrtg/_etc_mrtg.cfg_l) and it is
is only 0 seconds old ... Check your crontab.
(/etc/crontab and /var/spool/cron/root)

Has this something to do with the fact that my server has 2 NIC's?

[ Parent ]

#
Re: SNMP and MRTG on Sarge quick start
Posted by cswd (62.255.xx.xx) on Mon 6 Mar 2006 at 10:55
You don't need to add mrtg to cron. It's put in /etc/cron.d/ by dpkg when you install mrtg.

If your server has 2 NICs, you can uncomment the appropriate section in /etc/mrtg.cfg and it will generate graphs for the other interface.

Hope this helps,

[ Parent ]

#
Re: SNMP and MRTG on Sarge quick start
Posted by stephanhuiser (82.93.xx.xx) on Mon 6 Mar 2006 at 11:01
Yes, removing the cron entry solved my problem. Thanks!

[ Parent ]

#
Re: SNMP and MRTG on Sarge quick start
Posted by cswd (62.255.xx.xx) on Mon 6 Mar 2006 at 11:06
Glad to help :-)

[ Parent ]

#
Re: SNMP and MRTG on Sarge quick start
Posted by Anonymous (195.14.xx.xx) on Mon 6 Mar 2006 at 12:39
really nice one IF i want to get the disk,memory,cpu usage using snmp for all of my ststems i need to install snmp client or how to go about it

[ Parent ]

#
Re: SNMP and MRTG on Sarge quick start
Posted by Anonymous (216.46.xx.xx) on Mon 6 Mar 2006 at 14:01
You might want to try munin.
Configuring mrtg for anything custom (e.g. CPU monitoring) is a total pain in the ass, whereas munin comes with a bevy of preconfigured plugins and is very simple to extend. It also doesn't require that you mess with SNMP, which can be risky.

[ Parent ]

#
Re: SNMP and MRTG on Sarge quick start
Posted by Anonymous (80.109.xx.xx) on Mon 6 Mar 2006 at 15:48
mrtg can use more the snmp especially own scripts or just the output of commands. as a starting point here are my config stanzas for CPU&co (all found somewhere on the net and adapted): 
# Memory Utilization
# -------------------
Target[guinan_memory]: `free | awk '/buffers\/cache/ {print $3; print $4}'`
Options[guinan_memory]: nopercent,gauge,noinfo,growright, unknaszero
Title[guinan_memory]: Memory Utilization
PageTop[guinan_memory]: Memory Utilization on guinan
MaxBytes[guinan_memory]: 385476
kMG[guinan_memory]: k,M,G,T,P,X
YLegend[guinan_memory]: RAM Utilization
ShortLegend[guinan_memory]:
Legend1[guinan_memory]: Free guinan_memory
Legend2[guinan_memory]: Used guinan_memory
Legend3[guinan_memory]: Maximum free guinan_memory
Legend4[guinan_memory]: Maximum used guinan_memory
LegendI[guinan_memory]: Free mem
LegendO[guinan_memory]: Used mem
WithPeak[guinan_memory]: ymw
Unscaled[guinan_memory]: dwmy
Directory[guinan_memory]: system

# cpu-time                                         
Target[guinan-cpu_time]: `/usr/bin/awk '/cpu /{OFMT="%d"; print $2+$3; print $2+$3+$4; print $5/100/
60/60/24 " days"; print "guinan"}'Processor stats
MaxBytes[guinan-cpu_time]: 100
Options[guinan-cpu_time]: growright,nopercent 
LegendI[guinan-cpu_time]:  user: 
LegendO[guinan-cpu_time]:  total: 
Ylegend[guinan-cpu_time]: %
ShortLegend[guinan-cpu_time]: %
Legend1[guinan-cpu_time]: Time spent in user mode
Legend2[guinan-cpu_time]: Time spent in user mode + time spent in system mode
Legend3[guinan-cpu_time]: Maximum occurance of time spent in user mode
Legend4[guinan-cpu_time]: Maximum occurance of (time spent in user mode + time spent in system mode)
Unscaled[guinan-cpu_time]: dwmy
Directory[guinan-cpu_time]: system

# users
Target[guinan-users]: `who | wc -l | awk '{OFMT="%d"; print $1; print "0"}'`
Title[guinan-users]: Logged in users on guinan
PageTop[guinan-users]: Logged in users on guinan
MaxBytes[guinan-users]: 10
Options[guinan-users]: growright,nopercent,gauge
LegendI[guinan-users]:  users: 
LegendO[guinan-users]: 
Ylegend[guinan-users]: count
ShortLegend[guinan-users]:  user(s) 
Legend1[guinan-users]: Number of logged in users
spent in system mode)
Directory[guinan-users]: system

# space on hda1
Target[guinan-hda1]: `df -l | awk '/hda1/ {OFMT="%d"; print $3*1024; print $4*1024}'`
Title[guinan-hda1]: Disk usage: /dev/hda1 on guinan
PageTop[guinan-hda1]: Disk usage: /dev/hda1 on guinan (/)
MaxBytes[guinan-hda1]: 8217464832
Options[guinan-hda1]: growright,gauge
LegendI[guinan-hda1]:  used: 
LegendO[guinan-hda1]:  available: 
Ylegend[guinan-hda1]: byte
ShortLegend[guinan-hda1]: byte
Legend1[guinan-hda1]: Used space
Legend2[guinan-hda1]: Available space
Legend3[guinan-hda1]: Maximum used space
Legend4[guinan-hda1]: Maximum available space
Unscaled[guinan-hda1]: dwmy
Directory[guinan-hda1]: system
HTH, gregor

[ Parent ]

#
Re: SNMP and MRTG on Sarge quick start
Posted by jamesriden (203.114.xx.xx) on Sun 19 Mar 2006 at 01:16

I couldn't get

Target[guinan-cpu_time]: `/usr/bin/awk '/cpu /{OFMT="%d"; print $2+$3; print $2+$3+$4; print $5/100/60/60/24 " days"; print "guinan"}'Processor stats

to work for me - I had to replace the command with

`vmstat | tail -1 | /usr/bin/awk '/ /{OFMT="%d"; print $13; print $14;}'`

which gives user and system times, I think. Remember to fix the legends as well if you do this.

[ Parent ]

#
Re: SNMP and MRTG on Sarge quick start
Posted by Anonymous (80.109.xx.xx) on Mon 20 Mar 2006 at 23:32
sorry, this was a copy&paste error. the lines read in the original file as follows: 
Target[guinan-cpu_time]: `/usr/bin/awk '/cpu /{OFMT="%d"; print $2+$3; print $2+$3+$4; print $5/100/60/60/24 " days"; print "guinan"}'
Title[guinan-cpu_time]: Processor stats at home
gregor

[ Parent ]

#
Posted by Anonymous (80.109.xx.xx) on Mon 20 Mar 2006 at 23:37
*ARGL* still wrong. this editor seems to hate me. last try: 
Target[guinan-cpu_time]: `/usr/bin/awk '/cpu /{OFMT="%d"; print $2+$3; print $2+$3+$4; print $5/100/60/60/24 " days"; print "guinan"}' < /proc/stat`
Title[guinan-cpu_time]: Processor stats at home
/* the problem was the < */

[ Parent ]

#
Re: SNMP and MRTG on Sarge quick start
Posted by Anonymous (213.77.xx.xx) on Mon 6 Mar 2006 at 14:06
Wow, I was waiting for such a how-to. I've followed all the steps and now it works.
But what should be done if I want more graphs? My Debian-box is a gateway and router (NAT and so on) for a local network. It would be great to have such traffic-graphs for all computers in LAN.
Any suggestions? I think that I should tinker with /etc/mrtg.conf ?

[ Parent ]

#
Re: SNMP and MRTG on Sarge quick start
Posted by cswd (62.255.xx.xx) on Mon 6 Mar 2006 at 17:22
Edit /etc/mrtg.cfg ... all your interfaces were detected by cfgmaker so if you uncomment the relevant sections it will generate graphs for them.

[ Parent ]

#
Re: SNMP and MRTG on Sarge quick start
Posted by lee (193.82.xx.xx) on Mon 6 Mar 2006 at 15:58
[ View Weblogs ]

I don't know if it's changed in the last few years, but the processing overhead from generating new graphs became quite noticable with mrtg when more than a certain number of machines were being monitored. I started to use cricket at that point, which draws the graphs on demand.

Also, if all you want is network card interface counters via snmp, tinysnmp is quite good. You install the agent, and the module for the network mib, make a change to the IP address it listens on and the ranges that can query in the config file, restart and you're away.

apt-get install tinysnmp-agent tinysnmp-module-interfaces

It's a shame there aren't more modules available, the NIC and UPS modules seem to be the only useful ones in debian.

[ Parent ]

#
Posted by cswd (62.255.xx.xx) on Mon 6 Mar 2006 at 17:23
You DO need a dedicated NIC server/admin machine when you have REAL stats to monitor. I have a box that does backups, VPN, mrtg and notification.

[ Parent ]

#
Re: SNMP and MRTG on Sarge quick start
Posted by Anonymous (82.211.xx.xx) on Mon 6 Mar 2006 at 19:55
Thanks for this nice article - I read it just at the time when I need it. Althought I am more interested in rrdtool grapher it was very nice to read something on this topic here. And user's comments was very helpful too.
Thanks guys!

[ Parent ]

#
MRTG without SNMP
Posted by niol (82.239.xx.xx) on Mon 6 Mar 2006 at 20:50
[ View Weblogs ]

For those who only want bandwidth monitoring, this is achievable without SNMP.

MRTG comes with scripts that count stuff. By periodically running those scripts, you can have a graph. Here is my mrtg.conf with the call to the relevant script (mrtg-ip-acct part of the mrtgutils packages if I recall correctly) :

WorkDir: /var/www/site.org/stats
WriteExpires: Yes
Options[_]: growright

Title[^]: Traffic Analysis for

Target[eth1]: `/usr/bin/mrtg-ip-acct eth1`
MaxBytes1[eth1]: 1250000
MaxBytes2[eth1]: 125000
Title[eth1]: WAN Network interface
YLegend[eth1]: Traffic
PageTop[eth1]: Internet traffic analysis

And the corresponding cron entry should be in /etc/cron.d.

[ Parent ]

#
Re: SNMP and MRTG on Sarge quick start
Posted by Anonymous (195.23.xx.xx) on Tue 7 Mar 2006 at 10:53
Althoug it is a nice article, I prefer to use simple shell scripts to pull all the data that i want. Then after getting the data I use another shell to read the data and put it in a better solution then MRTG, RRDTOOl.
Cacti uses RRDTOOl
The new RRDTOOL version support trends and other features which makes the analisys easy.

[ Parent ]

#
Re: SNMP and MRTG on Sarge quick start
Posted by pepo (200.55.xx.xx) on Thu 9 Mar 2006 at 07:13
I wanna monitor the bandwidht of each user of my LAN that is using the gateway, Do I can use MRTG for this?. Maybe it is no the good place to make this question, sorry.

[ Parent ]

#
Re: SNMP and MRTG on Sarge quick start
Posted by Anonymous (62.255.xx.xx) on Thu 9 Mar 2006 at 07:15
Not easy to do that AFAIK.

[ Parent ]

#
Re: SNMP and MRTG on Sarge quick start
Posted by stoffell (81.83.xx.xx) on Fri 10 Mar 2006 at 12:48
You should do this on your gateway.
Try ZorbIPTraffic on www.atout.be, it does this.
But there are many other ways.. (also check out ntop)

cheers

[ Parent ]

#
nice reading, also check mrts
Posted by stoffell (81.83.xx.xx) on Fri 10 Mar 2006 at 12:50
Nice reading! Also have a look on the MRTS website: http://apt-get.dk/mrts/

It uses MRTG and RRDTool to create cool statistics and history of your interfaces. Nice to see the usage per day/week/month/year.. cheers!

[ Parent ]

#
Re: SNMP and MRTG on Sarge quick start
Posted by karl07071 (62.49.xx.xx) on Sun 19 Mar 2006 at 13:51
So i am new to linux and Debian, so forgive me if i misssed some crutial points here as i am what is affectionatly known as a newbie (insult or new nick name, to tell my friends about not sure)However in my delight to learn Debian Linux, i attempted some of the articles and so far got two out of four working.This MRTG SNMP article said it was easy so there is one little thing i appear to have missed and that is how do i access the results of my labour the article said something about Browsing to http://your-server-name/mrtg, when through out this configuration the mention nothing about creating a server to accommodate this, so i suppose i need to ask what did i mis and how do i fix it. By the way all worked well until this part so i cannot see the results of the MRTG. (That is newbies for you, not in the zone as yet) so help.

[ Parent ]

#
Re: SNMP and MRTG on Sarge quick start
Posted by Anonymous (62.255.xx.xx) on Sun 19 Mar 2006 at 15:02
If you have a web server, browse to http://127.0.0.1/mrtg/ ... If not, install apache2 using
apt-get install apache2

and then do it. Sorry I missed that!

[ Parent ]

#
Re: SNMP and MRTG on Sarge quick start
Posted by karl07071 (62.49.xx.xx) on Mon 20 Mar 2006 at 10:22
most excellent information dude, "BIG THANKS" absolutly loving it, worked first time, using http://127.0.0.1/mrtg in a web browser.


However the question remains, If i wanted to move this into an apache web server, whats the process? or do i get that from an apache config document like this one for the mrtg?

and if i wanted to add other graphs for other equipment on the lan now that i have this one working, is it the same process?

Sorry about the additional question, but (nubie getting into the zone albeit, slowly)

[ Parent ]

#
Re: SNMP and MRTG on Sarge quick start
Posted by bacula (86.135.xx.xx) on Sat 25 Mar 2006 at 20:18
[ View Weblogs ]
i am following this tutorial to configure mrtg in my system
mrtg for debian.i want to mainly monitor cpu,temperature and disk activity for my server how to i do that

[ Parent ]

#
Posted by Anonymous (217.150.xx.xx) on Wed 12 Apr 2006 at 11:05
I'll post a follow-up on how to do more advanced stuff with MRTG such as server status monitoring and additional equipment shortly. This is a LARGE topic so I will try and keep it simple but I'm not promising anything! CSWD ( http://www.cswd.co.uk/ )

[ Parent ]

#
Re: SNMP and MRTG on Sarge quick start
Posted by Anonymous (217.150.xx.xx) on Wed 12 Apr 2006 at 11:03
FYI I have removed my mrtg stats from the link in this article.

[ Parent ]

#
Re: SNMP and MRTG on Sarge quick start
Posted by Anonymous (193.11.xx.xx) on Sat 28 Oct 2006 at 15:07
Thanks a bundle, this worked like a charm. If only everything in life was so painless... :-)

The quality of the guides on this site is great I think, they give you a kickstart and a nudge in the right direction which often is all that is needed.

[ Parent ]

#
Re: SNMP and MRTG on Sarge quick start
Posted by Anonymous (80.175.xx.xx) on Tue 31 Oct 2006 at 23:58
Glad it works for you! Enjoy!

Part 2 coming up shortly - been so busy recently - too busy to even recover my password on here :(

- cswd

[ Parent ]

#
Re: SNMP and MRTG on Sarge quick start
Posted by Anonymous (202.70.xx.xx) on Fri 4 May 2007 at 13:30
I am getting problem with creating MRTG for tracking the bandwidth Traffic. Will anybody provide me the scripts and hints for watching the bandwidth Traffic.

[ Parent ]

#
Re: SNMP and MRTG on Sarge quick start
Posted by Anonymous (208.46.xx.xx) on Wed 22 Apr 2009 at 21:15
As of Lenny, assuming you don't want to do your network monitoring all on the same server that's being monitored, you'll need to edit /etc/defaults/snmpd and replace 127.0.0.1 with 0.0.0.0 in the snmpd command line to cause snmpd to listen on all interfaces.

By default, it listens only on the loopback interface which is useless for monitoring multiple machines.

[ Parent ]