Filesystems (ext3, reiser, xfs, jfs) comparison on Debian Etch
Posted by hansivers on Fri 21 Apr 2006 at 11:10
There are a lot of Linux filesystems comparisons available but most of them are anecdotal, based on artificial tasks or completed under older kernels. This benchmark essay is based on 11 real-world tasks appropriate for a file server with older generation hardware (Pentium II/III, EIDE hard-drive).
Since its initial publication, this article has generated a lot of questions, comments and suggestions to improve it. Consequently, I'm currently working hard on a new batch of tests to answer as many questions as possible (within the original scope of the article). Results will be available in about two weeks (May 8, 2006) Many thanks for your interest and keep in touch with Debian-Administration.org! Hans
Why another benchmark test?
I found two quantitative and reproductible benchmark testing studies using the 2.6.x kernel (see References). Benoit (2003) implemented 12 tests using large files (1+ GB) on a Pentium II 500 server with 512MB RAM. This test was quite informative but results are beginning to aged (kernel 2.6.0) and mostly applied to settings which manipulate exclusively large files (e.g., multimedia, scientific, databases).
Piszcz (2006) implemented 21 tasks simulating a variety of file operations on a PIII-500 with 768MB RAM and a 400GB EIDE-133 hard disk. To date, this testing appears to be the most comprehensive work on the 2.6 kernel. However, since many tasks were "artificial" (e.g., copying and removing 10 000 empty directories, touching 10 000 files, splitting files recursively), it may be difficult to transfer some conclusions to real-world settings.
Thus, the objective of the present benchmark testing is to complete some Piszcz (2006) conclusions, by focusing exclusively on real-world operations found in small-business file servers (see Tasks description).
Test settings
- Hardware
- Processor : Intel Celeron 533
- RAM : 512MB RAM PC100
- Motherboard : ASUS P2B
- Hard drive : WD Caviar SE 160GB (EIDE 100, 7200 RPM, 8MB Cache)
- Controller : ATA/133 PCI (Silicon Image)
- OS
- Debian Etch (kernel 2.6.15), distribution upgraded on April 18, 2006
- All optional daemons killed (cron,ssh,saMBa,etc.)
- Filesystems
- Ext3 (e2fsprogs 1.38)
- ReiserFS (reiserfsprogs 1.3.6.19)
- JFS (jfsutils 1.1.8)
- XFS (xfsprogs 2.7.14)
Description of selected tasks
- Operations on a large file (ISO image, 700MB)
- Copy ISO from a second disk to the test disk
- Recopy ISO in another location on the test disk
- Remove both copies of ISO
- Operations on a file tree (7500 files, 900 directories, 1.9GB)
- Copy file tree from a second disk to the test disk
- Recopy file tree in another location on the test disk
- Remove both copies of file tree
- Operations into the file tree
- List recursively all contents of the file tree and save it on the test disk
- Find files matching a specific wildcard into the file tree
- Operations on the file system
- Creation of the filesystem (mkfs) (all FS were created with default values)
- Mount filesystem
- Umount filesystem
The sequence of 11 tasks (from creation of FS to umounting FS) was run as a Bash script which was completed three times (the average is reported). Each sequence takes about 7 min. Time to complete task (in secs), percentage of CPU dedicated to task and number of major/minor page faults during task were computed by the GNU time utility (version 1.7).
RESULTS
Partition capacity
Initial (after filesystem creation) and residual (after removal of all files) partition capacity was computed as the ratio of number of available blocks by number of blocks on the partition. Ext3 has the worst inital capacity (92.77%), while others FS preserve almost full partition capacity (ReiserFS = 99.83%, JFS = 99.82%, XFS = 99.95%). Interestingly, the residual capacity of Ext3 and ReiserFS was identical to the initial, while JFS and XFS lost about 0.02% of their partition capacity, suggesting that these FS can dynamically grow but do not completely return to their inital state (and size) after file removal.
Conclusion : To use the maximum of your partition capacity, choose ReiserFS, JFS or XFS.
File system creation, mounting and unmounting
The creation of FS on the 20GB test partition took 14.7 secs for Ext3, compared to 2 secs or less for other FS (ReiserFS = 2.2, JFS = 1.3, XFS = 0.7). However, the ReiserFS took 5 to 15 times longer to mount the FS (2.3 secs) when compared to other FS (Ext3 = 0.2, JFS = 0.2, XFS = 0.5), and also 2 times longer to umount the FS (0.4 sec). All FS took comparable amounts of CPU to create FS (between 59% - ReiserFS and 74% - JFS) and to mount FS (between 6 and 9%). However, Ex3 and XFS took about 2 times more CPU to umount (37% and 45%), compared to ReiserFS and JFS (14% and 27%).
Conclusion : For quick FS creation and mounting/unmounting, choose JFS or XFS.
Operations on a large file (ISO image, 700MB)
The initial copy of the large file took longer on Ext3 (38.2 secs) and ReiserFS (41.8) when compared to JFS and XFS (35.1 and 34.8). The recopy on the same disk advantaged the XFS (33.1 secs), when compared to other FS (Ext3 = 37.3, JFS = 39.4, ReiserFS = 43.9). The ISO removal was about 100 times faster on JFS and XFS (0.02 sec for both), compared to 1.5 sec for ReiserFS and 2.5 sec for Ext3! All FS took comparable amounts of CPU to copy (between 46 and 51%) and to recopy ISO (between 38% to 50%). The ReiserFS used 49% of CPU to remove ISO, when other FS used about 10%. There was a clear trend of JFS to use less CPU than any other FS (about 5 to 10% less). The number of minor page faults was quite similar between FS (ranging from 600 - XFS to 661 - ReiserFS).
Conclusion : For quick operations on large files, choose JFS or XFS. If you need to minimize CPU usage, prefer JFS.
Operations on a file tree (7500 files, 900 directories, 1.9GB)
The initial copy of the tree was quicker for Ext3 (158.3 secs) and XFS (166.1) when compared to ReiserFS and JFS (172.1 and 180.1). Similar results were observed during the recopy on the same disk, which advantaged the Ext3 (120 secs) compared to other FS (XFS = 135.2, ReiserFS = 136.9 and JFS = 151). However, the tree removal was about 2 times longer for Ext3 (22 secs) when compared to ReiserFS (8.2 secs), XFS (10.5 secs) and JFS (12.5 secs)! All FS took comparable amounts of CPU to copy (between 27 and 36%) and to recopy the file tree (between 29% - JFS and 45% - ReiserFS). Surprisingly, the ReiserFS and the XFS used significantly more CPU to remove file tree (86% and 65%) when other FS used about 15% (Ext3 and JFS). Again, there was a clear trend of JFS to use less CPU than any other FS. The number of minor page faults was significantly higher for ReiserFS (total = 5843) when compared to other FS (1400 to 1490). This difference appears to come from a higher rate (5 to 20 times) of page faults for ReiserFS in recopy and removal of file tree.
Conclusion : For quick operations on large file tree, choose Ext3 or XFS. Benchmarks from other authors have supported the use of ReiserFS for operations on large number of small files. However, the present results on a tree comprising thousands of files of various size (10KB to 5MB) suggest than Ext3 or XFS may be more appropriate for real-world file server operations. Even if JFS minimize CPU usage, it should be noted that this FS comes with significantly higher latency for large file tree operations.
Directory listing and file search into the previous file tree
The complete (recursive) directory listing of the tree was quicker for ReiserFS (1.4 secs) and XFS (1.8) when compared to Ext3 and JFS (2.5 and 3.1). Similar results were observed during the file search, where ReiserFS (0.8 sec) and XFS (2.8) yielded quicker results compared to Ext3 (4.6 secs) and JFS (5 secs). Ext3 and JFS took comparable amounts of CPU for directory listing (35%) and file search (6%). XFS took more CPU for directory listing (70%) but comparable amount for file search (10%). ReiserFS appears to be the most CPU-intensive FS, with 71% for directory listing and 36% for file search. Again, the number of minor page faults was 3 times higher for ReiserFS (total = 1991) when compared to other FS (704 to 712).
Conclusion : Results suggest that, for these tasks, filesystems can be regrouped as (a) quick and more CPU-intensive (ReiserFS and XFS) or (b) slower but less CPU-intensive (ext3 and JFS). XFS appears as a good compromise, with relatively quick results, moderate usage of CPU and acceptable rate of page faults.
OVERALL CONCLUSION
These results replicate previous observations from Piszcz (2006) about reduced disk capacity of Ext3, longer mount time of ReiserFS and longer FS creation of Ext3. Moreover, like this report, both reviews have observed that JFS is the lowest CPU-usage FS. Finally, this report appeared to be the first to show the high page faults activity of ReiserFS on most usual file operations.
While recognizing the relative merits of each filesystem, only one filesystem can be install for each partition/disk. Based on all testing done for this benchmark essay, XFS appears to be the most appropriate filesystem to install on a file server for home or small-business needs :
- It uses the maximum capacity of your server hard disk(s)
- It is the quickest FS to create, mount and unmount
- It is the quickest FS for operations on large files (>500MB)
- This FS gets a good second place for operations on a large number of small to moderate-size files and directories
- It constitutes a good CPU vs time compromise for large directory listing or file search
- It is not the least CPU demanding FS but its use of system ressources is quite acceptable for older generation hardware
While Piszcz (2006) did not explicitly recommand XFS, he concludes that "Personally, I still choose XFS for filesystem performance and scalability". I can only support this conclusion.
References
Benoit, M. (2003). Linux File System Benchmarks.
Piszcz, J. (2006). Benchmarking Filesystems Part II. Linux Gazette, 122 (January 2006).
I use ext3 because most tools are written for it, and everything Linux supports it.
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm
The first column names the filesystem tested. The second column records the total time (in seconds) it took to run the filesystem benchmarking software bonnie++ (Version 1.93c). The third column records the total number of megabytes needed to store 655 megabytes of raw data.
SMALLER is better.
| FILESYSTEM | TIME | DISK USAGE |
|---|---|---|
| REISER4 (lzo) | 1,938 | 278 |
| REISER4 (gzip) | 2,295 | 213 |
| REISER4 | 3,462 | 692 |
| EXT2 | 4,092 | 816 |
| JFS | 4,225 | 806 |
| EXT4 | 4,408 | 816 |
| EXT3 | 4,421 | 816 |
| XFS | 4,625 | 799 |
| REISER3 | 6,178 | 793 |
| FAT32 | 12,342 | 988 |
| NTFS-3g | >10,414 | 772 |
Each test was preformed 5 times and the average value recorded. SMALLER is better.
The Reiser4 filesystem clearly had the best test results.
The FAT32 filesystem had the worst test results.
The bonnie++ tests were preformed, with the following parameters:
bonnie++ -n128:128k:0
[ Parent | Reply to this comment ]
These days, tools and integration have been setup quite nicely by the distribution maintainers, and a fsck-interface is used by all of the ones I've tried anyway.
Ofcourse, if you want to play with more advanced options, dump filesystems or do anything out of the ordinary your findings may be different -- I would not know since I rarely, if ever, use these features.
On the other hand, IMO it rarely matters which filesystem you use anyway. I would challenge anybody to guess the filesystem running on a light to medium loaded desktop or server. Differences (in speed of mature common journaling filesystems) really are rather small for general use, and it's not until you have very specific tasks to be done or very i/o loaded systems to be managed that the choice of journaling filesystem becomes a real issue.
I believe that XFS has upcoming (or perhaps already has some) support in FreeBSD, though. FreeBSD has ext2 (read) support too. And IIRC, there was a (non-microsoft, obviously) driver adding ext2 (hence, ext3) support to Windows -- if you run that OS and want to allow it to touch your nice Linux system, that is. I suppose that falls under compatibility too.
[ Parent | Reply to this comment ]
- What did you use to compare the times? What tools? Which commands?
- It was not mentioned the fact that Ext3 reserves 5% of the HD to the root user. (see http://ubuntuforums.org/showthread.php?t=215177)
- Some points in the article like the other is better. That to generic to say that other FS took less or spend less. (e.g.: The ReiserFS used 49% of CPU to remove ISO, when other FS used about 10%). What other FS took about 10% of CPU?
The rest of the article I found good. My tip is trying to use a better server, like a Core Duo or an AMD X2, or even an Xeon or Opteron, since we`re talking of bussiness servers (but that`s good enough if you don`t have one to test). Maybe, a good thing is to test in SATA drivers or SCSI...
Note: I use ext3 mostly because of compatibility. I like new stuff, but I don`t like to play with FS's.
[ Parent | Reply to this comment ]
I use ReiserFS because it's the only filesystem that supports shrinking a filesystem - veeeery useful with LVM! - JFS, XFS, ... - backup - resize - restore - with 1 TB (that's not that much anymore nowadays) - that's a joke. Measuring the performance of this operation is missing in the article.
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
The tasks selected here were not intended to be comprehensive, since I was focusing more on adding some new hard data to Piszcz's excellent benchmark series. I will surely add your suggestion in a follow-up testing.. Thanks!
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
You stated that all filesystems were created using default values. So ext3 loses approx. 5% of its capacity because of its reserved-blocks feature. For a fileserver you would create your data-partition without reserved blocks, as it is not needed there.
Or was this taken into account already?
Cheers,
Johannes
[ Parent | Reply to this comment ]
/Nafallo
[ Parent | Reply to this comment ]
Why would you want to make a fileserver to perform badly? The 5% default reserve is in part to have some slack for use by 'root', but mostly because when a filesystem is nearly full its performance becomes very bad as fragmentation increases nonlinearly.
My experience is that 'ext3' requires at least 10% free space to perform decently over time (the 5% default is the absolute minimum that should be done) and 20-30% free space reserve is a lot better.
The problem is indeed intrinsic: as the filesystem nears 100% usage the chances of finding contiguous or nearby free blocks when writing or extending a file becomes a lot smaller. This applies to both extent based filesystems like JFS and XFS and to block based filesystems like 'ext3' (even if usually extent based filesystems do a bit better).
[ Parent | Reply to this comment ]
Intrinsic perhaps, but highlights what is missing, such as how filesystems address the issue. Part of the cost of writing the ReiserFS is all that messing with binary trees, the report doesn't attempt to understand, or address, why Reiser thinks it is worth doing all this (admittedly a lot of people said it was just too expensive to be worth trying).
Very little in the way of "aging" the filesystem.
Nothing on consistency, or limits, or features.
I think the test choice is not great, filesystem creation, mounting, and manipulating ISOs are generally not time critical tasks (well if you don't use NTFS they aren't!). I'd happily use a file system that take 100 times longer to create than any of those tested if it conveyed other discernable benefits, and it takes my CD writer over 5 minutes to write a full ISO, so a few second here or there matters not at all to me on manipulating ISOs.
Be interesting to see also how representative people think 7500 files being 1.9GB is. My understanding was that mean file size, whilst on the way up, hadn't got to that sort of size yet. Certainly I have 15GB (df -h) of files on this box, and just under 0.5 million files (find / -type f), clocking in just under 32KB average file size for my file size, rather than the 253KB used in the test. Perhaps someone is hoarding ISOs?
It is well known there is a cost-benefit trade off in ReiserFS that means it performs relatively less well on larger files than XFS. So something like mean file size is likely to explain the difference between the results here, and of other authors, on the performance of ReiserFS.
I'd also prefer to see more edge cases examined -- what happens when 100,000 emails are delivered, and then sorted, and a selection deleted, to a single maildir? For most people it is probably performing in a sensible manner under these edge cases, that matters far more than if it takes 130 or 135s to copy 7000 files.
I'd like to see blocking I/O cases, and similar examined, email delivery being a classic, and fairly easy to test.
Hardest of all to test, I want to know that the filesystem journalling "does what it says on the tin", and have someone pull the plug in the middle of these transaction, and see nothing is corrupted, and that everything is in a consistent state, and how long the recovery to that consistent state takes, and that it is automatic.
Why no "bonnie++" statistics -- I'd have thought as a test it was trivial to run, and might show up something, even if the I/O types measured are a tad artificial.
Then again I appreciate these tests take a lot of time and effort to do.
Nothing here will shake my own choice of ReiserFS for most general purpose filesystems, it has a good performance on real world benchmarks, and is the most mature of the journalling file systems presented. Although I'm looking at XFS for a project, but not because of its performance, but because of other features it brings to the table.
[ Parent | Reply to this comment ]
There's a myth abroad that modern drives detect falling supply voltage and do stuff to protect the media. (You might read of using the motor as a generator to provide power to "park" the heads.) If it was ever true, it was only in high-end drives, and not any more. Even making sure data really is physically on the disk surface before reporting the write complete, something we like to think of as the basic promise, is no longer widely supported, though the drives will claim otherwise. In practice you need up to a few seconds of power after a crash to drain sectors from the cache to the disk surface.
The mantra is, if reliability matters, replicate and use battery ("UPS") power.
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
I personally lost half of my homedir to XFS because it does only metadata journaling. Sure, all my files were there because of the journaling, but they were filled with binary zeroes. Oh and yes, the XFS FAQ says that this may happen (well at least it did when I was searching for an explanation of that behaviour back then).
I really don't want to use any "journaling" filesystem that does not journal my data, because then its worthless. I don't need a filesystem that has a clean tree and can be mounted if I lose half of my data in it. ReiserFS was pretty good at garbling file contents ("WTF is my mp3 playlist now that videos I just downloaeded?") too before they had data journaling.
[ Parent | Reply to this comment ]
Just to let you know, Hitachi does uses this technology to move heads into proper place if power loss detected.However I have no idea if drive is able to flush buffers in this scenario.Perhaps they able, since Hitachi is intentionally(?) restricts write buffer size while read buffer size allowed to be whole size of installed RAM IC.
[ Parent | Reply to this comment ]
I'm fairly new to linux and may well have been doing something wrong, but my box regularly had the power pulled on it (my area used to be prone to power dips and I couldn't afford a UPS).
When I was using ext3 it sometimes took me a few hours to get the system to even boot, because it refused while there were errors. Once I switched to ReiserFS all those problems went away.
In a single user environment, which is realist enough for me, i'd chose the FS that allows me to recover quickest over an FS that might take a second or two longer to do something. (Can't remember the last time I copied 7000 files, if ever.)
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
Once I switched to ReiserFS all those problems went away.I once used ReiserFS on a fileserver and after a broken PSU took the server down, it was the data that went away, because the tree ReiserFS uses was corrupted.
The recovery utilities supplied for ReiserFS tried their best to recover the filesystem, but in the end I had to restore from the backup of the night before.
After this incident, I decided to give Ext3 and XFS a try. While XFS seems to be the more modern filesytem, it does lack the ability to shrink, which is a real problem in times of LVM2 and Software-RAID.
Ext3 hasn't let me down ever since. It's disaster recovery tools are the maturest of all the filesystems tested (in addition to being included in every live/recovery CD on the planet) and it's online resizing capabilities really go together well with virtualized (as well as real) infrastructure.
On a side note, I also found Ext3 to be the most tolerant filesystem for use on "flaky" hardware. If, for example, a part of the binary tree used in ReiserFS happens to land on a defective sector of the harddisk, then it's bye-bye time for your entire FS. Ext3, on the other hand, will cope quite well and allows for full recovery using one of its redundantly stored superblocks.
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
I added these data to replicate other previous observations about filesystem creation and mounting time.
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
Pete .
[ Parent | Reply to this comment ]
Every time you choose to leave your computer turned on, you are choosing to disregard a finite chance that pollution will make this planet uninhabitable for future generations. That's not such a "simple answer" any more is it?
Wasting electricity despoils the commons. Turn the computer off when you aren't using it.
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
It's so much more useful to leave the machine on all the time!
Perhaps you are shutting it down to reduce noise, in which case, I commend quietpc.com to you - I can hear birdsong over mine, with the windows shut.
[ Parent | Reply to this comment ]
Why keep it running if you don't need it or can easily wake it up if you need it?
[ Parent | Reply to this comment ]
Slow the CPU (especially if it's a P4 or old Athlon) and hibernate the monitor. That itself will save a good amount of energy.
Or, better, host your home server on a passively-cooled Via system. Then you can shut-down your PC any time you want, while the server stays up, sipping watts.
[ Parent | Reply to this comment ]
Actually, SWSUSP2 works well enough now that I hibernate all of the machines at home (except the server) when they're not going to be used, such as overnight. Hibernation on my HP laptop is almost infallible -- I've got an "uptime" of over a month, hibernating once or twice (and sometimes more) every day -- and, while it takes a good minute to go into hibernation, it comes out of it within 35 seconds... that's from hitting the power switch to having my KDE desktop back up.
I just wish S3 worked as well, and that the kernel folks would adopt SWSUSP2, which works so much better than the default hibernate mechanism.
--- SER
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
File system creation time can't really be an issue to anyone, I guess.
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
The problem is that unless the IO subsystem supports mailboxing and tagged queueing, which are only available in practice on SCSI and SCSI/ATA host adapters (3ware and up), multiple concurrent accesses have awful performance.
However there are already some filesystem speed tests for suitable IO subsystems, alluded to by some other comment, for example:
http://ext2.SourceForge.net/2005-ols/ols-presentation-html/img38. html
BTW, in this graph the JFS performance comes out badly, I think that an older version of JFS was used that had excessive locking like 'ext3' for most of its life.
There are more links to filesystem speed tests here:
http://WWW.sabi.co.UK/Notes/anno05-3rd.html#050911
[ Parent | Reply to this comment ]
NMP
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
* during a file tree copy, pull the plug on the machine. (could be simulated by running the test under vmware and killing the virtual machine) - then check how well it the fs recovers the data - which files got corrupted (if any), how long it takes to fix (replaying journals, etc).
* call your initial tree t0. make a new tree apporximate the same size called t1. for a concurrency test:
cp -a t0 t2 & # one tree
cp -a t1 t3 & # other tree
cp -a t0 t4 & # merge the trees
cp -a t1 t4 &
Hrm, should probably test mixed concurrency (deletes too!) so:
cp -a t0 t5 && rm -rf t5 &
I'm sure there's more to be added, maybe this will give you some ideas.
[ Parent | Reply to this comment ]
In these experiments the test variable was disk configuration rather than file system. A similar test across different file systems might produce a worthwhile indication of comparative reliability.
I suspect write caching would need to be disabled in the disk system to prevent corruptions of the kind being investigated in the link above from affecting the results. This would have an impact on absolute performance, but relative measurements could still be made.
[ Parent | Reply to this comment ]
Yank the plug while multiple processes are updating the disks. See what happens.
Repeat 8 or 10 times.
Yes, it's manual and time-consuming.
[ Parent | Reply to this comment ]
With ReiserFS 3 on the other hand, I've got two such events and both times It managed to somehow completely destroy multiple files which were not even open at the time of incident.
(Yes this is anecdotal evidence, but I'm not using it anymore because of there incidents)
[ Parent | Reply to this comment ]
It just isn't fun to see the filesystem break and read a publicity message about being able to ask questions for $25... (ReiserFS)
[ Parent | Reply to this comment ]
Get a recent ReiserFS and mount it with data=journal!
[ Parent | Reply to this comment ]
An intesting (and simple) test would be to simulate power failures during copy/delete file operations (large ISO file and files tree) and see how each FS handles each situation. But I'm aware that this is only a small part of real data integrity testing..
If anybody has seen hard data about FS reliability, feel free to post a link here. I would be very interested to investigate this, in order to produce more comprehensive and real-world benchmarks.
[ Parent | Reply to this comment ]
However, I have found one annoying feature with xfs: whenever there is power failure then all open text files are filled with "^@^@^@^@^@^@^@^@...". You can easily replicate this by opening file /etc/fstab to emacs and then unplug the power cord. Why /etc/fstab.... well, then you know why I find this feature REALLY annoying. So, this powerfailure test would be the First on my test list for real world servers.
Anyway, I enjoyed reading your article. And being on professional researcher myself I know that there is always room for improvement. Looking forward reading the new comparison from you.
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
mail / # mount|grep -i xfs
/dev/hda3 on / type xfs (rw,noatime,logbufs=8,logbsize=32768,ihashsize=65567)
/dev/mapper/vg-usr on /usr type xfs (rw,nodev,noatime,logbufs=8,logbsize=32768,ihashsize=65567)
/dev/mapper/vg-home on /home type xfs (rw,nosuid,nodev,noatime,usrquota,grpquota,logbufs=8,logbsize=327 68,ihashsize=65567)
/dev/mapper/vg-opt on /opt type xfs (rw,nodev,noatime,logbufs=8,logbsize=32768,ihashsize=65567)
/dev/mapper/vg-var on /var type xfs (rw,nodev,noatime,usrquota,grpquota,logbufs=8,logbsize=32768,ihas hsize=65567)
/dev/mapper/vg-tmp on /tmp type xfs (rw,noexec,nosuid,nodev,noatime,usrquota,grpquota,logbufs=8,logbs ize=32768,ihashsize=65567)
/dev/hda1 on /boot type xfs (rw,noatime,logbufs=8,logbsize=32768,ihashsize=65567)
mail / # ls -lah /boot/grub/*xfs*
-rw-r--r-- 1 root root 11K Jul 1 2005 /boot/grub/xfs_stage1_5
mail / # grub --version
grub (GNU GRUB 0.96)
mail / #
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
I have xfs on all my nfs servers. OS is SuSE 9.2/9.3/10.0.
[ Parent | Reply to this comment ]
- Peder
[ Parent | Reply to this comment ]
You said "While recognizing the relative merits of each filesystem, an system administrator has no choice but to install only one filesystem...". I don't understand why you believe there is or should be this sort of restriction.
Can't the administrator decide to 'partition' usage onto different volumes, using different fs types, based on their performance for the usage?
For example, I might create a volume to hold users homes, expecting many small files while requiring maximum speed, and so choose to use XFS, while a volume to hold large files (video, audio, backups, still images, etc.), I might choose to use JFS instead.
Note that I'm not advocating or even suggesting that the above is in some way an optimal setup, it's just an 'off the top of my head' example. The question is "Why shouldn't I be able to do this sort of thing if I so choose?" Is there something I'm missing?
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
For example, did you unmount the relevant partition before every single operation? You don't say, but if you did not (and not many people know that this is essential) your results are largely meaningless.
For far more sensible, documented and informative tests look at mine here:
http://WWW.sabi.co.UK/Notes/anno05-3rd.html#050908
and in a few entries around that date. Some amusing updates here:
http://WWW.sabi.co.UK/Notes/anno05-3rd.html#050913
http://WWW.sabi.co.UK/Notes/anno06-2nd.html#060416
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
I used to think much the same, but over time I discovered that I'd rather use JFS across the board (except for filesystems that need to be accessible from MS Windows, where I use 'ext2' as there is an excellent filesystem for it).
The first reason is that 'ext3' performance is awesome when the filesystem has just been created and loaded, but degrades very badly over time while JFS degrades significantly but a lot less:
http://WWW.sabi.co.UK/Notes/anno06-2nd.html#060416
The second reason is that probably because of some happenstance 'dir_index' can slow down things pretty significantly:
http://WWW.sabi.co.UK/Notes/anno05-4th.html#051204
A rather less significant advantage of JFS is that since it uses extents and dynamically allocated inodes it usually uses a lot less space for metadata, often like 3-5% of the total filesystem space.
[ Parent | Reply to this comment ]
--------
Felipe Sateler
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
grep -i acl config-2.6.15:
CONFIG_XFS_POSIX_ACL=y
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
The biggest problem however is not making or formatting a filesystem, it is how long it takes to 'fsck' it, and how much memory is necessary.
Times of over two months to 'fsck' a filesystem have been reported for 'ext3' and XFS sometimes requires more than 4GB of memory to run 'fsck' (it is possible to create and use an XFS filesystem on a system with a 32 bit that can only be 'fsck'ed on a 64 bit CPU, and at least one case has actually happened).
The basic problem is that while very large filesystems using JFS or XFS (or very recent 'ext3') perform well on RAID storage, because they take advantage of the parallel nature of the underlying storage system, 'fsck' is single threaded in every Linux file system design that I have seen. Bad news.
More details here:
http://www.sabi.co.uk/Notes/anno05-4th.html#051012
http://www.sabi.co.uk/Notes/anno05-4th.html#051009
I am very surprised that your experience is that «jfs just wasn't stable enough», perhaps you may want to report to the JFS mailing list, as the authors of JFS are very responsive to reports of instaibility, and usually find a fix pretty quickly.
As to FC5 support, all Red Hat systems only support 'ext3', at least officially and in the installer, but after installation you can use any of the filesystems included in the kernel. I typically install to a small temporary partition which is 'ext3' formatted, and then convert it to JFS by copying its contents over to the real ''root'' partition which is JFS formatted.
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
Fig
[ Parent | Reply to this comment ]
The shread command works by writing random data, zeros, and ones over and over to the spots on the disk that the file you want to shred was located. The hopes are that with enough writes, the data will actually be overwritten on the disk. (The head of the hard drive varies a small amout as it traces its path over the disk, so the data might not be completely erased).
The problem is, journaling file systems write data to the journal before they write it to the final location on the disk. So shredding the file blocks on the disk, an attacker might be able to recover data from wherever on the disk the journal is located, even if the data blocks are unreadable.
The real issue is that shredding a file even on ext2 does not always work, because modern hard drives sometimes transparently remap bad sectors on the disk... so what the operating system thinks is the location on the disk it originally wrote mysecret.txt to, the drive might have moved it. An attacker could still read data from the "bad" sector using the right tools.
Realisticly, shred should never be relied on. Using dm-crypt to encrypt a full filesystem is a much better solution, and with today's CPUs power, the performance is small enough trade off for secrecy.
For more information about securly destroying data, you can read the paper TKS1 on this page, which is really interesting. (Scroll down to section 3 on page 4)
[ Parent | Reply to this comment ]
From man shred:
"... Note that shred relies on a very important assumption: that the file
system overwrites data in place. This is the traditional way to do things, but
many modern file system designs do not satisfy this assumption.
[snip ]...
In the case of ext3 file systems, the above disclaimer applies (and shred is thus
of limited effectiveness) only in data=journal mode, which journals file data in
addition to just metadata. In both the data=ordered (default) and data=writeback
modes, shred works as usual. "
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
Thats rather wide of the mark: most filesystem are supposed to recover A CONSISTENT STATE of the METADATA ONLY from such events.
'ext3' additionally can make an attempt at recovering the contents of files too, if ordered or data journaling is enabled.
However the proper way to ensure data (as opposed to metadata) recoverability is to ensure the application handles that, using atomic data transactions, because that's the only way, and even if 'ext3' often succeeds blindly, that is not the right way.
Large scale filesystems like JFS and XFS, designed for mission critical applications, don't do any attempt at data recovery, because indeed that should be handled by the applications themselves.
Many people who don't understand this then complain that then these two filesystems cause loss of data...
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
In all the cases you mentioned, your chances would indeed be best if you indeed had your data on an Ext3 filesystem, since it's the only one not using some binary tree structure to manage where your data is stored.
The problem of today's hardware is all the caching that's going on at various levels. The application can't really tell whether a certain file has really been written to a block on the harddisk, because all of that is completely hidden away in some HAL.
I think that Sun's approach with it's ZFS filesystem is suitable to tackle this challenge. It uses end-to-end checksums to detect file corruption at the harddisk right through to the application level. Too bad it isn't GPL'd, so we'll hardly see much of it in the Linux world.
[ Parent | Reply to this comment ]