06/29/2000
Are you running an Intel Linux system with at least one (E)IDE hard drive?
Wouldn't it be neat if there were a magical command to instantly double the I/O performance of your disks? Or, in some cases, show 6 to 10 times your existing throughput?
Did you ever just wonder how to tell what kind of performance you're getting on your "tricked-out" Linux box?
Don't overlook hdparm(icon_cool.gif. If you've never heard of it, don't worry. Most people I've talked to haven't either. But if you're running an IDE/Linux system (as many folks are,) you'll wonder how you ever got this far without it. I know I did.
What's the big deal?
So, you've got your brand-new UltraATA/66 EIDE drive with a screaming brand-new controller chipset that supports multiple PIO modes and DMA and the leather seat option and extra chrome... But is your system actually taking advantage of these snazzy features? The hdparm(icon_cool.gif command will not only tell you how your drives are performing, but will let you tweak them out to your heart's content.
Now before you get too excited, it is worth pointing out that under some circumstances, these commands CAN CAUSE UNEXPECTED DATA CORRUPTION! Use them at your own risk! At the very least, back up your box and bring it down to single-user mode before proceeding.
With the usual disclaimer out of the way, I'd like to point out that if you are using current hardware (i.e. your drive AND controller AND motherboard were manufactured in the last two or three years), you are at considerably lower risk. I've used these commands on several boxes with various hardware configurations, and the worst I've seen happen is the occasional hang, with no data problems on reboot. And no matter how much you might whine at me and the world in general for your personal misfortune, we all know who is ultimately responsible for the well-being of YOUR box: YOU ARE. Caveat Fair Reader.
Now, then. If I haven't scared you away yet, try this (as root, preferably in single-user mode):
hdparm -Tt /dev/hda
You'll see something like:
/dev/hda:
Timing buffer-cache reads: 128 MB in 1.34 seconds =95.52 MB/sec
Timing buffered disk reads: 64 MB in 17.86 seconds = 3.58 MB/sec
What does this tell us? The -T means to test the cache system (i.e., the memory, CPU, and buffer cache). The -t means to report stats on the disk in question, reading data not in the cache. The two together, run a couple of times in a row in single-user mode, will give you an idea of the performance of your disk I/O system. (These are actual numbers from a PII/350 / 128M Ram / newish EIDE HD; your numbers will vary.)
But even with varying numbers, 3.58 MB/sec is PATHETIC for the above hardware. I thought the ad for the HD said something about 66MB per second!!?!? What gives?
Well, let's find out more about how Linux is addressing your drive:
hdparm /dev/hda
/dev/hda:
multcount = 0 (off)
I/O support = 0 (default 16-bit)
unmaskirq = 0 (off)
using_dma = 0 (off)
keepsettings = 0 (off)
nowerr = 0 (off)
readonly = 0 (off)
readahead = 8 (on)
geometry = 1870/255/63, sectors = 30043440, start = 0
These are the defaults. Nice, safe, but not necessarily optimal. What's all this about 16-bit mode? I thought that went out with the 386! And why are most of the other options turned off?
Well, it's generally considered a good idea for any self-respecting distribution to install itself in the kewlest, slickest, but SAFEST way it possibly can. The above settings are virtually guaranteed to work on any hardware you might throw at it. But since we know we're throwing something more than a dusty, 8-year-old, 16-bit multi-IO card at it, let's talk about the interesting options:
multcount: Short for multiple sector count. This controls how many sectors are fetched from the disk in a single I/O interrupt. Almost all modern IDE drives support this. The man page claims:
When this feature is enabled, it typically reduces operating system overhead for disk I/O by 30-50%. On many systems, it also provides increased data throughput of anywhere from 5% to 50%.
I/O support: This is a big one. This flag controls how data is passed from the PCI bus to the controller. Almost all modern controller chipsets support mode 3, or 32-bit mode w/sync. Some even support 32-bit async. Turning this on will almost certainly double your throughput (see below.)
unmaskirq: Turning this on will allow Linux to unmask other interrupts while processing a disk interrupt. What does that mean? It lets Linux attend to other interrupt-related tasks (i.e., network traffic) while waiting for your disk to return with the data it asked for. It should improve overall system response time, but be warned: Not all hardware configurations will be able to handle it. See the manpage.
using_dma: DMA can be a tricky business. If you can get your controller and drive using a DMA mode, do it. But I have seen more than one machine hang while playing with this option. Again, see the manpage (and the example on the next page)!
Turbocharged
So, since we have our system in single-user mode like a good little admin, let's try out some turbo settings:
hdparm -c3 -m16 /dev/hda
/dev/hda:
setting 32-bit I/O support flag to 3
setting multcount to 16
multcount = 16 (on)
I/O support = 3 (32-bit w/sync)
Great! 32-bit sounds nice. And some multi-reads might work. Let's re-run the benchmark:
hdparm -tT /dev/hda
/dev/hda:
Timing buffer-cache reads: 128 MB in 1.41 seconds =90.78 MB/sec
Timing buffered disk reads: 64 MB in 9.84 seconds = 6.50 MB/sec
WOW! Almost double the disk throughput without really trying! Incredible.
But wait, there's more: We're still not unmasking interrupts, using DMA, or even a using decent PIO mode! Of course, enabling these gets riskier. (Why is it always a trade-off between freedom and security?) The man page mentions trying Multiword DMA mode2, so:
hdparm -X34 -d1 -u1 /dev/hda
...Unfortunately this seems to be unsupported on this particular box (it hung like an NT box running a Java app.) So, after rebooting it (again in single-user mode), I went with this:
hdparm -X66 -d1 -u1 -m16 -c3 /dev/hda
/dev/hda:
setting 32-bit I/O support flag to 3
setting multcount to 16
setting unmaskirq to 1 (on)
setting using_dma to 1 (on)
setting xfermode to 66 (UltraDMA mode2)
multcount = 16 (on)
I/O support = 3 (32-bit w/sync)
unmaskirq = 1 (on)
using_dma = 1 (on)
And then checked:
hdparm -tT /dev/hda
/dev/hda:
Timing buffer-cache reads: 128 MB in 1.43 seconds =89.51 MB/sec
Timing buffered disk reads: 64 MB in 3.18 seconds =20.13 MB/sec
20.13 MB/sec. A far cry from the miniscule 3.58 we started with...
By the way, notice how we specified the -m16 and -c3 switch again? That's because it doesn't remember your hdparm settings between reboots. Be sure to add the above line to your /etc/rc.d/* scripts once you're sure the system is stable (and preferably after your fsck runs; having an extensive fs check run with your controller in a flaky mode may be a good way to generate vast quantities of entropy, but it's no way to administer a system. At least not with a straight face...)
Now, after running the benchmark a few more times, reboot in multi-user mode and fire up X. Load Netscape. And try not to fall out of your chair.
In conclusion
This is one of those interesting little tidbits that escapes many "seasoned" Linux veterans, especially since one never sees any indication that the system isn't using the most optimal settings. (Gee, all my kernel messages have looked fine....) And using hdparm isn't completely without risk, but is well worth investigating.
And it doesn't stop at performance: hdparm lets you adjust various power saving modes as well. See the hdparm(icon_cool.gif for the final word.
Many thanks to Mark Lord for putting together this nifty utility. If your particular distribution doesn't include hdparm (usually in /sbin or /usr/sbin), get it from the source at http://metalab.unc.edu/pub/Linux/system/hardware/
Happy hacking! icon_lol.gif
