What will you miss when this site closes?





204 votes ~ 6 comments

This site will turn read-only at the end of September 2017.

Monitoring your filesystem for unauthorised change

Posted by Steve on Tue 30 Nov 2004 at 14:43

If you're running a stable server and are worried about an intruder modifying your system binaries to install new corrupted versions you should be using a filesystem integrity checker.

There are several available as part of Debian's stable and unstable archive.

The most widely known integrity checker is tripwire, but several other packages are available to do the same job including integrit and aide.

All these tools work in the same way, and it's mostly a matter of personal preference which one you choose to install.

When they are first installed you use them to build up a database of all your important files, and a corresponding checksum of their contents.

Later you can recompute the checksums of the binaries and compare them against those you stored in your initial database. This will allow you to detect any binaries, or files, on your local filesystem which have been modified.

It is important that you store the database of the filesystem securely - such that any malicious intruder couldn't update it to hide their tracks.

A good way of doing this if you have physical access is to burn it onto CD-ROM as this allows you to mount it without the ability to write to it.

However each time you legitimately update your system you must rebuild your pristine database - so this might get expensive fast.

As a simple guide we'll walk through installing both integrit and aide, the latter seems to be the most popular integrity checker available.

Installing integret is very straightforward. Download and install it via apt-get and you'll be presented with a couple of simple questions.

apt-get install integrit

When it is installed you must be careful to not tell the software to update or create its database - because we've not configured it yet. All the other questions may be safely answered with the defaults.

Once installed you'll find a configuration file /etc/integrit/integrit.conf.

This configuration file contains a list of directories, or paths, which are checked.

Every file beneath the named directory will be checksumed using the SHA-1 hash, and its details will be stored in the integret database located at /var/lib/integrit.

The configuration file contains a list of example directories along with a brief explaination of how to add new entries.

A minimal configuration for my machine looks like this:

#
# Global settings
#
root=/
known=/var/lib/integrit/known.cdb
current=/var/lib/integrit/current.cdb


#
#  Ignore '!' the following directories because we don't care if their contents are modified.
#
!/mnt
!/dev
!/etc
!/home
!/lost+found
!/proc
!/tmp
!/usr/local
!/usr/src

Once this is setup you can create the initial database:

integrit -C /etc/integrit/integrit.conf -u

This saves the current state of the system into the file /var/lib/integrit/current.cdb, we need to move this into the known state - and also take a copy offsite.

mv /var/lib/integrit/current.cdb /var/lib/integrit/known.cdb 

Mailing a copy of this file offsite to a safe location is useful as it allows you to test again later - even if you think your database might have been modified by a local user.

To check the filesystem for changes we can now run:

integrit -C /etc/integrit/integrit.conf -c

As you've just created a pristine database you should see no errors.

To test that the system is working run:

touch /bin/ls  # Modify a file
integrit -C /etc/integrit/integrit.conf -c

This time you should see an error message:

changed: /bin/ls   m(20020318-151001:20041130-142618) c(20031107-102841:20041130-142618)

(m in this case is the modification date of the file, c being the creation date).

The Debian package will mail you every day if files have changed - and even if they haven't. There is a cron job setup by the file /etc/cron.daily/integrit. You can edit that file if you only wish to see an email in the case of differences, the comments explain how to do so:

    # * UNCOMMENT the two following lines marked with `# !' if you don't
    # * want to receive reports if no mismatches were found

    # ! if [ '$(echo '$output' | egrep -v '^integrit: ')' ]; then
        message=$(echo '$message' && echo '$output')
    # ! fi

This overview really showed you the kind of thing that you will have to do with any integrity checking system:

  • Create an initial database.
  • Move it somewhere safe. (So that it can be used if you don't trust the local copy).
  • Run regular checks of the current system against that database.

All the systems we've mentioned so far, aide, integrit, and tripwire use exactly this mode of operation.

Looking at aide next we can work through the same example.

First of all install it, and when prompted decline the opertunity to create the initial database:

apt-get install aide

aide is configured by the file /etc/aide/aide.conf and the process is mostly the same as that shown for integrit already.

The configuration file defines a list of checks, such as the following:

Binlib = p+i+n+u+g+s+b+m+c+md5+sha1

Here we see the check called Binlib is defined as a combination of different tests from the following table:

# Here are all the things we can check - these are the default rules
#
#p:      permissions
#i:      inode
#n:      number of links
#u:      user
#g:      group
#s:      size
#b:      block count
#m:      mtime
#a:      atime
#c:      ctime
#S:      check for growing size
#md5:    md5 checksum
#sha1:   sha1 checksum
#rmd160: rmd160 checksum
#tiger:  tiger checksum
#R:      p+i+n+u+g+s+m+c+md5
#L:      p+i+n+u+g
#E:      Empty group
#>:      Growing logfile p+u+g+i+n+S

There are a number of tests defined for different purposes, such as ConfFiles designed to cover things in /etc, Logs for logfiles, etc.

Then these tests are applied to a group of directories.

So my previous example covering most of the important directories looks like this for aide:

# Binaries
/bin            Binlib
/sbin           Binlib
/usr/bin        Binlib
/usr/sbin       Binlib
/usr/local/bin  Binlib
/usr/local/sbin Binlib
/usr/games      Binlib

# Libraries
/lib            Binlib
/usr/lib        Binlib
/usr/local/lib  Binlib

# Logfiles
/var/log$       StaticDir
/var/log        Logs

# Things to ignore
!/dev
!/proc
!/mnt
!/usr/src
!/usr/doc
!/usr/share/doc

Once this is done you can intialise the database, with the following command:

aideinit

The database, by default, will be placed in /var/lib/aide/aide.db.new. If you're happy with the output you can copy it to the real location for running tests against:

mv /var/lib/aide/aide.db.new /var/lib/aide/aide.db

As before we'll modify a file and then run a test:

touch /bin/ls
aide --check

This gives the following output:

AIDE found differences between database and filesystem!!
Start timestamp: 2004-11-30 14:39:45
Summary:
Total number of files=11247,added files=0,removed files=0,changed files=1


Changed files:
changed:/bin/ls

Detailed information about changes:

File: /bin/ls
  Mtime    : 2004-11-30 14:26:18               , 2004-11-30 14:39:39
  Ctime    : 2004-11-30 14:26:18               , 2004-11-30 14:39:39

As you can see this is more readable than the example we showed previously with integrit, but this is offset by the trickier setup required.

If you're happy to acknowlege the change just re-run the aideinit command and move the new database into the live location - this will cause future checks to be error free.

I hope this was a useful comparision between two local filesystem integrity checkers.

 

 


A different approach to integrity checking
Posted by docelic (213.202.xx.xx) on Mon 3 Jan 2005 at 18:52
If you're interested in verifying files without keeping a master "comparison" database around, see PKI-based tools such as bsign or elfsign. They can only sign binary files, however.

[ Parent | Reply to this comment ]

Re: Monitoring your filesystem for unauthorised ch
Posted by nathanbullock (161.184.xx.xx) on Tue 22 Feb 2005 at 17:50
[ View Weblogs ]
In the article you mention

>> It is important that you store the database of the filesystem securely - such that any malicious intruder couldn't update it to hide their tracks.

>> A good way of doing this if you have physical access is to burn it onto CD-ROM as this allows you to mount it without the ability to write to it.

Wouldn't just storing the md5sum of the database be good enough? I just store this md5sum on the desktop of one of my other machines (not the server that aide is running on). This way when I ssh in I do an md5sum on the aide database make sure it matches then run aide -check.

Second. The one problem that you don't mention is that by storing this db elsewhere you keep people from modifying it to hide their tracks. Couldn't they just modify the aide binary to hide their tracks so that it gives the appropriate answers?

[ Parent | Reply to this comment ]

Re: Monitoring your filesystem for unauthorised ch
Posted by Steve (82.41.xx.xx) on Tue 22 Feb 2005 at 18:13
[ View Weblogs ]

Yes if they can modify the binary then you would not spot anything, the perfect solution is to burn the binary to the CD-ROM too - and use that for the checking.

Or boot from a known-good kernel, Knoppix etc.

Then any kernel rootkits will be unable to disguise the tampering.

Steve
-- Steve.org.uk

[ Parent | Reply to this comment ]

Re: Monitoring your filesystem for unauthorised ch
Posted by Seaslug (128.2.xx.xx) on Sat 7 May 2005 at 04:22
[ View Weblogs ]
All well and good, but I don't see documentation readily available for checking Master Boot Record. How is this done?

[ Parent | Reply to this comment ]

Re: Monitoring your filesystem for unauthorised ch
Posted by Anonymous (81.181.xx.xx) on Wed 11 May 2005 at 13:44
Read some bytes from /deb/hda (or what are you using), save them, hash them, do what ever you want with them.

Then compare them everytime you desire with what is currently available in /dev/hda.

[ Parent | Reply to this comment ]

Re: Monitoring your filesystem for unauthorised ch
Posted by Anonymous (81.181.xx.xx) on Wed 11 May 2005 at 13:44
Read some bytes from /deb/hda (or what are you using), save them, hash them, do what ever you want with them.

Then compare them everytime you desire with what is currently available in /dev/hda.

[ Parent | Reply to this comment ]

Re: Monitoring your filesystem for unauthorised ch
Posted by Anonymous (128.2.xx.xx) on Wed 11 May 2005 at 22:17
Ah... dd if=/dev/hda of=MBRcopy bs=512 count=1

do diff between MBRcopy and new copy generated when compare is needed....

Can run simple script to do so at boot, for example, if diff fails, print "Big brother has modified your machine! Run away!", or sounds a klaxon, or sends insulting messages to various government agencies.....

[ Parent | Reply to this comment ]

Re: Monitoring your filesystem for unauthorised ch
Posted by Anonymous (81.181.xx.xx) on Mon 16 May 2005 at 22:00
You've got to hide it someway though...
Someone (an intruder) might check your init scripts to see anything suspcious (from his side of view).
Also he may change your file and you'll never see that you've got a modified MBR. (Read-only may be the best option)

Do you really need that? :D

[ Parent | Reply to this comment ]

Re: Monitoring your filesystem for unauthorised change
Posted by Anonymous (217.64.xx.xx) on Sun 25 Sep 2005 at 11:03
Mailing a copy of this file offsite to a safe location is useful as it allows you to test again later - even if you think your database might have been modified by a local user.

Won't simply doing md5sum "$known" and storing the checksum securely suffice?

[ Parent | Reply to this comment ]

Re: Monitoring your filesystem for unauthorised change
Posted by Steve (82.41.xx.xx) on Sun 25 Sep 2005 at 11:04
[ View Weblogs ]

Sure that would tell you that it has changed - but it won't tell you what has changed...

Steve
--

[ Parent | Reply to this comment ]