This site is now 100% read-only, and retired.

XML logo

Disk access per process
Posted by neofpo on Tue 19 Jun 2007 at 16:02
Tags: none.
I am trying to find a way to let my notebook's HD spin down to save battery, however something still uses it constantly and I can not find what it is.

I recently used Windows Vista and it has a very nice feacure: a top like disk access statistics. I can easily identify what process is using the disk and at what rate. I could manage to easily solve some performance issues on Vista using this tool.

The question is: is there a way Linux kernel export disk access usage per process? All I have seen until now is a system wide value, for a whole block device...

There is the brute force method of course, lsof + find by latest modification / access date... but that's not exactly what I want.

Ah, I know about laptop mode, but I wish to fix the cause of the problem (no more useless disk access).

 

Comments on this Entry

Re: Disk access per process
Posted by Anonymous (213.164.xx.xx) on Wed 20 Jun 2007 at 14:39
You can't do it.

[ Parent ]

Re: Disk access per process
Posted by dkg (216.254.xx.xx) on Thu 21 Jun 2007 at 03:45
[ View Weblogs ]
This is a pretty frustrating comment, Anonymous. You could at least provide a reference to someone backing up your claim here.

Also, there's no reason that one couldn't produce this information in principle, is there? What pieces are missing to getting this info? Is there a kernel interface reporting issue? Is /proc not providing statistics that it should? Are we missing a userland tool that reads and reports on the contents of /proc in a human-readable fashion? Or is there actually some reason that each physical disk access can't be legitimately associated with a single process?

The clearer the problem is, the better shot we have of solving it. I'd very much like to be able to do what neofpo is suggesting.

[ Parent ]

Maybe available in sysstat 7.1.5 (with kernel 2.6.20+)
Posted by dkg (216.254.xx.xx) on Thu 21 Jun 2007 at 03:57
[ View Weblogs ]
Just after making the reply above, i stumbled into the SYSSTAT homepage, which currently shows the following:
Saturday June 9th, 2007

Sysstat 7.1.5 released (development version).

At last, this version includes a long awaited feature: per-process I/O statistics, that you can now display with the command pidstat and its new option -d. Of course, you will need a recent Linux kernel for that (2.6.20+).

Version 7.1.5 doesn't appear to be in debian yet (and you'd need to be running the kernel from sid anyway):
[0 dkg@squeak ~]$ apt-cache policy sysstat
sysstat:
  Installed: 7.1.4-1
  Candidate: 7.1.4-1
  Version table:
 *** 7.1.4-1 0
        500 http://ftp.debian.org testing/main Packages
        200 http://ftp.debian.org unstable/main Packages
        100 /var/lib/dpkg/status
[0 dkg@squeak ~]$ 
But if you are running the latest kernel, you should try building the new package. Report back when you get a chance!

You might also want to file a wishlist bug against debian's sysstat package, asking for the new version.

[ Parent ]

Re: Maybe available in sysstat 7.1.5 (with kernel 2.6.20+)
Posted by neofpo (200.185.xx.xx) on Thu 21 Jun 2007 at 04:07
[ View Weblogs ]
Wow!! Thanks for the tip dude. I was running 2.6.18, looked at /proc, everywhere. Only at unstable...

I will check this feature on the next days, hope top incorporate it.

[ Parent ]

Re: Maybe available in sysstat 7.1.5 (with kernel 2.6.20+)
Posted by dkg (216.254.xx.xx) on Thu 21 Jun 2007 at 04:54
[ View Weblogs ]
No problem. If you have a chance to switch to the new kernel from unstable, i've just built an i386 version of sysstat 7.1.5 to try out. You can download it if you don't want to build it yourself (and you're running an i386 machine).

It built for me with no errors at all:

[0 dkg@squeak ~]$ mkdir -p src/sysstat
[0 dkg@squeak ~]$ cd src/sysstat
[0 dkg@squeak sysstat]$ apt-get source sysstat
Reading package lists... Done
Building dependency tree... Done
Need to get 234kB of source archives.
Get:1 http://ftp.debian.org testing/main sysstat 7.1.4-1 (dsc) [578B]
Get:2 http://ftp.debian.org testing/main sysstat 7.1.4-1 (tar) [208kB]
Get:3 http://ftp.debian.org testing/main sysstat 7.1.4-1 (diff) [25.4kB]
Fetched 234kB in 1s (118kB/s)                                  
dpkg-source: extracting sysstat in sysstat-7.1.4
dpkg-source: unpacking sysstat_7.1.4.orig.tar.gz
dpkg-source: applying ./sysstat_7.1.4-1.diff.gz
[0 dkg@squeak sysstat]$ (cd sysstat-7.1.4/ && uscan)
sysstat: Newer version (7.1.5) available on remote site:
  http://perso.orange.fr/sebastien.godard/sysstat-7.1.5.tar.gz
  (local version is 7.1.4)
sysstat: Successfully downloaded updated package sysstat-7.1.5.tar.gz
    and symlinked sysstat_7.1.5.orig.tar.gz to it
[1 dkg@squeak sysstat]$ tar xzf sysstat-7.1.5.tar.gz 
[0 dkg@squeak sysstat]$ cd sysstat-7.1.5
[0 dkg@squeak sysstat-7.1.5]$ zcat ../sysstat_7.1.4-1.diff.gz | patch -s -p1
[0 dkg@squeak sysstat-7.1.5]$ debuild -uc -us
and when debuild finished running, i had the new package available, which i could put online. If you're not running i386, you could probably do the same steps on your architecture to get a native package yourself. debuild and uscan are in the devscripts package, so other than that and build-essential, you'll only need to make sure you can satisfy the build-deps for sysstat:
[0 dkg@squeak sysstat]$ grep ^Build-Depends: sysstat-7.1.4/debian/control 
Build-Depends: debhelper (>= 5.0.7), gettext
[0 dkg@squeak sysstat]$ 
Please let me know if it works for you, because i've often wanted this feature myself. It might even be worth playing with the unstable kernel.

[ Parent ]

Re: Maybe available in sysstat 7.1.5 (with kernel 2.6.20+)
Posted by neofpo (200.185.xx.xx) on Thu 21 Jun 2007 at 15:00
[ View Weblogs ]
Thank you for the package, But I will not be able to use it. I played with the unstable kernel (and many other packages) and broke my Debian notebook system. I installed Ubuntu on it. It has kernel 2.6.20, I'll try it on here. My Desktop runs Debian Stable, that I am not going to change ;-)

[ Parent ]

Re: Disk access per process
Posted by madphilosopher (221.122.xx.xx) on Wed 27 Jun 2007 at 07:40
I've had good luck with atop and the atop kernel patches for 2.6.18 on Debian Etch. The patches add the necessary code to the Linux kernel to do per-process disk and network accounting. (I find it incredible that such a basic feature is missing from Linux!)

I follow the instructions below to patch my kernel:

* http://www.atconsultancy.nl/atop/kernpatch.html

And to build a kernel package:

* http://www.debian.org/releases/stable/i386/ch08s06.html.en

Once you're running the new kernel, the atop utility (available from apt) has a disk mode to show you per-process disk accesses.

--
Darren Paul Griffith
www.madphilosopher.ca

[ Parent ]

Re: Disk access per process
Posted by Anonymous (203.97.xx.xx) on Fri 29 Jun 2007 at 15:03
From memory I monitored all disk accesses and followed a trail which led me to insert

hdparm -B 255 /dev/hda

in /etc/rc.local

There may have been something added to call /etc/rc.local

This stops the disk spinning down every second or so which just can't be good.

[ Parent ]

Re: Disk access per process
Posted by neofpo (200.185.xx.xx) on Fri 29 Jun 2007 at 16:32
[ View Weblogs ]
My problem is the opposite: something does not let my HD spin down and laptop mode is not enough.

I still had no time to test this new feature to identify I/O by process. Think I will be able to do this at this weekend.

[ Parent ]