This site is now 100% read-only, and retired.

Monitoring network traffic with ngrep

Posted by Steve on Mon 7 Mar 2005 at 01:05

ngrep is a piece of software which is designed to mirror the standard pattern matching utility grep, although instead of matching patterns against text files it matches traffic passing over a network interface.

To use ngrep it helps if you're familiar with the GNU grep package.

grep is a standard utility you can expect to find on Unix and Linux systems. Put simply it allows you to search for text in files, the real power comes from its ability to handle regular expressions.

To use grep in a basic way is very simple, you merely invoke it with the text you wish to find, and the list of files you wish to search. For example you could list all the lines in your password file which contain the term "root" by running:

grep root /etc/passwd

This is only a simple, literal, match though and doesn't contain a regular expression (which is what grep was named for - it's name is an abreviation for "Global Regular Expression Print").

Simple regular expressions were known to all users of DOS who executed commands like:

copy *.txt ..

In that example "*.txt" is a pattern matching any file which ends in ".txt". A corresponding Unix example could be:

cp *.html /tmp

Both are similar and contain the notion of a wildcard - the "*" character meaning literally anything can match.

Full regular expressions have a lot of special characters which can allow particular things to match.

Here are some simple examples of the kind of things that can be used inside regular expressions :

.

The period (.) matches any single character.

[...]

The square brackets match a range of characters included inside them. For example [1-9] matches any one of the digits 0 - 9. The example [abf] matches any single character a,b, or f.

^

The carat symbol only matches at the start of a line.

$

The dollar symbol only matches at the end of a line.

?

The question mark matches any single character.

*

The star character matches the previous character or term zero or more times. So whilst . would match a single character .* would match anything - because the initial period matches any character, and the star causes the match to apply zero or more times.

+

The plus symbol matches something only if the previous match worked one or more times. So for example [0-9]+ matches only text which contains multiple digits.

With that simple list out of the way we can look at matching things in a more complex way than just literally.

Using three of the things we learnt we could search for all lines in a text file which consist only of numbers by using the following pattern:

^[0-9]+$

This uses the ^ symbol to match only at the start of the line things which consist only of digits ([0-9]) one or more times (+) which are at the end of a line ($).

These patterns apply to many of the standard Unix tools, sed, awk, and programming languages such as Perl and PHP.

There isn't really enough space here to do justice to the subject of regular expressions, but hopefully we've covered enough to help people who've never seen them before.

Installing ngrep is simple thanks to apt-get, simply run the following command as root:

apt-get install ngrep

This will download ngrep for you along with any libraries it requires which you don't already have installed.

Once it's installed you can start exploring the data passing through your network interface. As it requires the ability to sniff packets it will require root privileges to run.

The basic usage is:

ngrep [options] pattern [filter]

Options are explained in the manpage, and pattern is a regular expression of data to match on the network. The optional filter allows you to limit the matches to data going to or from a particular host or port.

You may well capture sensitive information so if you need permission to perform any network capturing you should obtain it before proceeding.

Surfing the web with a browser involves sending requests to remote servers using a protocol called HTTP, or HyperText Transfer Protocol.

This involves your browser sending requests to remote servers and reading and displaying the returned information, be it text graphics, or other items.

To see which files your browser is requesting you can run the following command whilst you are surfing:

ngrep -q '^GET .* HTTP/1.[01]'

This matches any packet on the network which consists of "GET" followed by any characters but ending in "HTTP/1.0" or "HTTP/1.1".

Accessing the index of this site results in the following output:

T 192.168.1.80:37331 -> 80.68.89.210:80 [AP]
  GET /index.cgi HTTP/1.1..Host: www.debian-administration.org..User-Agent: 
  Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.6) Gecko/20050301 Firefox/1
  .0.1 (Debian package 1.0.1-1)..Accept: text/xml,application/xml,applicatio
  n/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5..Accept-L
  anguage: en-us,en;q=0.5..Accept-Encoding: gzip,deflate..Accept-Charset: IS
  O-8859-1,utf-8;q=0.7,*;q=0.7..Keep-Alive: 300..Connection: keep-alive..

T 192.168.1.80:37333 -> 80.68.89.210:80 [AP]
  GET /css/gila-screen.css HTTP/1.1..Host: www.debian-administration.org..Us
  er-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.6) Gecko/20050301
   Firefox/1.0.1 (Debian package 1.0.1-1)..Accept: text/css,*/*;q=0.1..Accep
  t-Language: en-us,en;q=0.5..Accept-Encoding: gzip,deflate..Accept-Charset:
   ISO-8859-1,utf-8;q=0.7,*;q=0.7..Keep-Alive: 300..Connection: keep-alive..
  Referer: http://www.debian-administration.org/index.cgi

T 192.168.1.80:37333 -> 80.68.89.210:80 [AP]
  GET /css/gila-print.css HTTP/1.1..Host: www.debian-administration.org..Use
  r-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.6) Gecko/20050301 
  Firefox/1.0.1 (Debian package 1.0.1-1)..Accept: text/css,*/*;q=0.1..Accept
  -Language: en-us,en;q=0.5..Accept-Encoding: gzip,deflate..Accept-Charset: 
  ISO-8859-1,utf-8;q=0.7,*;q=0.7..Keep-Alive: 300..Connection: keep-alive..R
  eferer: http://www.debian-administration.org/index.cgi..

Here you can see three requests, one for the index page, and two more for different CSS files, non-printable characters such as newline characters have been replaced by periods in the output to make it easier to read.

These requests were matched just because they contained the text we were looking for, sending an email which contained the same text would also have matched, because we didn't specify that we wished to limit our matching to a specific port.

If we wished, for example, to only match requests sent to a particular port, or host we could have done so as follows:

# Match only requests going  to port 80
ngrep -q '^GET .* HTTP/1.[01]' 'port 80' 

# Match only requests going to the destination 'Slashdot.org'
ngrep -q '^GET .* HTTP/1.[01]' 'host slashdot.org' 

# Match only ping requests
ngrep -q '.' 'icmp'

When it comes to tracking down unusual activity it's very useful to use ngrep, as it shows you both the source and the destination of traffic which matches.

Once I had an ISDN dialup modem connected to a host which was setup to dial out everytime traffic was sent to it. Over time it became obvious that it was dialling up every 30 minutes due to some broken cronjob, but it wasn't clear where the traffic was coming from.

Using ngrep made this simple:

ngrep -q '.' 'host dialup.machine'

In case you're wondering the '-q' flag causes us to only see packets which match our patterns. By default ngrep will show a '#' mark for each packet it ignores.

 

 


Re: Monitoring network traffic with ngrep
Posted by Anonymous (203.122.xx.xx) on Sat 16 Apr 2005 at 04:51
I illustrated some uses of ngrep in an article here. It talks about monitoring the connectivity given by a cablewallah ISP in New Delhi (these are guys who string up ethernet cable around the neighbourhood to give you the last-mile connectivity in the form of a shared LAN).

eg: ngrep was a handy tool for detecting possible spammers with:

ngrep -iqtl -d eth1 'mail from' tcp port 25 >> possiblespamlist

[ Parent ]