Is posible to tune HDD performance under linux?

[hikaruichijo]

Hi,
I have this server:
Supermicro 370dl3 motherboard, two PIII 1000 cpus, 1.5gb SDRAM,two nics, the integrated one and an intel mt1000 PCI-64, two sata cards, one sil3112a PCI and one 3ware 9500s 4lp PCI-64, and six HDDs connected to then.
I have IPFIRE instaled on it at it is working as a router/firewall/proxy and NAS.
First of all, I don't have a lot of experience with linux neither with this old servers.
My problem is that for example when I copy files between two diferent hdds with mignight commander it only reaches around a max on 11mbs second in the case of big files.
I don't know if the machine can't give more because it's old hardware or if I have something wrong.
Maybe people with more experience can tell me if the new sata hdds are limited by the old P3 hardware and that 11mbs is the maximum posible or if I can do something to improbe the perfomance of the hdds.
Thanks.

[stj]

Re: Is posible to tune HDD performance under linux?

pci bus is probably the limiting factor, but it can run faster than that i'm sure.
in linux you can change the cache handling, and mess with the drives using various tools
https://duckduckgo.com/?q=linux+hard...t-wt&kv=1&kh=1

[hikaruichijo]

Re: Is posible to tune HDD performance under linux?

Thanks, I'm going to change my search engine to duckduckgo, google did'nt show that kind of results.
I also thought that it could be faster than that, specially the 3ware card because it is connected to a pci-64 100mhz bus.....
Let's see if I'm smart enought to find a solution

[hikaruichijo]

Re: Is posible to tune HDD performance under linux?

I did a test and found that midnight commander is using around 70% of the cpu when moving files, maybe that is a limiting factor?

[stj]

Re: Is posible to tune HDD performance under linux?

pci is 33Mhz
pci-64 is 64bit - so 64bit at 33Mhz

there are pci buses that run at 66Mhz, but i'v only seen them on ppc based mac's and some server boards.

i dont think there is a pci variant that runs 100Mhz

[hikaruichijo]

Re: Is posible to tune HDD performance under linux?

Sorry, it was my mistake, it's 66mhz no 100, sorry.
These are the specs of the board,
http://supermicro.com/products/mothe.../LE/370DL3.cfm
Now i'm following your advice and triying thins with the hdparm command.

[ratdude747]

Re: Is posible to tune HDD performance under linux?

pci is 33Mhz
pci-64 is 64bit - so 64bit at 33Mhz

there are pci buses that run at 66Mhz, but i'v only seen them on ppc based mac's and some server boards.

i dont think there is a pci variant that runs 100Mhz

PCI-X can go up to 133MHz (depends on the motherboard, many only have one 1-3 PCI-X slots rated for such).

[ChaosLegionnaire]

Re: Is posible to tune HDD performance under linux?

using 70% cpu? it shouldnt be doing that. make sure its using dma and not on pio mode.

it could also be a problem with midnight commander. try a different linux file manager. it could also be doing some verification or file hashing as it copies the file to make sure the file is transferred correctly and when it comes to large files, it leaks memory due to the size of the file during the verification/hashing process thus it slows down.

i have also read an issue whereby there were issues with certain types of filesystem drivers leaking memory and slowing down after awhile. are the drives using ntfs filesystem? those leak memory and slow down after awhile.

[stj]

Re: Is posible to tune HDD performance under linux?

what filesystem is it using - not FAT i hope!

[hikaruichijo]

Re: Is posible to tune HDD performance under linux?

I'm using ext4. I've been toying with the filesysten cache and it now starts copying at 20-21mbs but it starts slowing down after a minute or so and drops againg to 11-12 mbs
I've found that the ipfire distro has virtual memory swapines disabled by default.
I don't now of another comand line file managers, I'm going to search around the net. any recomendation?
Thanks.

[eccerr0r]

Re: Is posible to tune HDD performance under linux?

What do these return (use your particular hdd)?

hdparm -V
hdparm -ivtT /dev/sda

I recall getting buffered read speeds on my Tualatin Celeron 1200 exceeding 50MB/sec on single IDE disks when idle... I forget cached read speeds however.

[RJARRRPCGP]

Re: Is posible to tune HDD performance under linux?

PCI-X can go up to 133MHz (depends on the motherboard, many only have one 1-3 PCI-X slots rated for such).

PCI-X is 160 MB/s, IIRC and an Adaptec 29160 SCSI card with regular PCI slot will be 133 MB/s.

[hikaruichijo]

Re: Is posible to tune HDD performance under linux?

I already did this tests but anyway this is the output of hddparm.
[root@ipfire ~]# hdparm -V
hdparm v9.45
[root@ipfire ~]# hdparm -ivtT /dev/sda

/dev/sda:
multcount = 16 (on)
IO_support = 1 (32-bit)
readonly = 0 (off)
readahead = 256 (on)
geometry = 60801/255/63, sectors = 976773168, start = 0

Model=TOSHIBA MQ01ABD050, FwRev=AX001A, SerialNo=83UJPJDPT
Config={ Fixed }
RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=0
BuffType=unknown, BuffSize=8192kB, MaxMultSect=16, MultSect=16
CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=976773168
IORDY=on/off, tPIO={min:120,w/IORDY:120}, tDMA={min:120,rec:120}
PIO modes: pio0 pio1 pio2 pio3 pio4
DMA modes: sdma0 sdma1 sdma2 mdma0 mdma1 mdma2
UDMA modes: udma0 udma1 udma2 udma3 udma4 *udma5
AdvancedPM=yes: unknown setting WriteCache=enabled
Drive conforms to: Unspecified: ATA/ATAPI-3,4,5,6,7

* signifies the current active mode

Timing cached reads: 366 MB in 2.00 seconds = 182.97 MB/sec
Timing buffered disk reads: 184 MB in 3.02 seconds = 60.87 MB/sec
[root@ipfire ~]# hdparm -ivtT /dev/sdd

/dev/sdd:
HDIO_DRIVE_CMD(identify) failed: Invalid argument
readonly = 0 (off)
readahead = 256 (on)
geometry = 243201/255/63, sectors = 3907029168, start = 0
HDIO_GET_IDENTITY failed: Invalid argument
Timing cached reads: 404 MB in 2.00 seconds = 201.95 MB/sec
Timing buffered disk reads: 190 MB in 3.00 seconds = 63.27 MB/sec


As far as I undesrtand everythin seen OK but I'm still getting slow speeds.
sda is connected to the sil3112 and sdd is conected to the 9500s.

I think is a problem with ipfire and in the oficial forum they stated that they don't want to support old hardware anymore.

[eccerr0r]

Re: Is posible to tune HDD performance under linux?

I don't think it's a disk tuning problem, the speeds look like around what they should be. Maybe a little slow but in the ballpark. Chances are that the cpu is doing too much on something else. Hitting bad sectors? Any SMART data? Any i/o error messages in dmesg? Is there a specific disk that's having problems? Are you copying one disk to another?

Fragmentation on those files? (filefrag filename)

[SteveNielsen]

Re: Is posible to tune HDD performance under linux?

What kind of performance do you get copying a large file(s) from the terminal command line?

[hikaruichijo]

Re: Is posible to tune HDD performance under linux?

Checked the smart and everithing seens ok, no relocations or pending relocations.
Tried moving one 2gb file with the mv command and it took about 3 minutes with means around 11mbs.
But in dmesg I've found this:
[28848.364422] 3w-9xxx: scsi1: ERROR: (0x03:0x0101): Invalid command opcode:opcode=0x85.
[28848.366136] 3w-9xxx: scsi1: ERROR: (0x03:0x0101): Invalid command opcode:opcode=0x85.
[28848.551842] 3w-9xxx: scsi1: ERROR: (0x03:0x0101): Invalid command opcode:opcode=0x85.
[28848.553566] 3w-9xxx: scsi1: ERROR: (0x03:0x0101): Invalid command opcode:opcode=0x85.
[28848.729445] 3w-9xxx: scsi1: ERROR: (0x03:0x0101): Invalid command opcode:opcode=0x85.
[28848.731249] 3w-9xxx: scsi1: ERROR: (0x03:0x0101): Invalid command opcode:opcode=0x85.
[28848.908294] 3w-9xxx: scsi1: ERROR: (0x03:0x0101): Invalid command opcode:opcode=0x85.
[28848.910253] 3w-9xxx: scsi1: ERROR: (0x03:0x0101): Invalid command opcode:opcode=0x85.
[28849.010285] 3w-9xxx: scsi1: ERROR: (0x03:0x0104): SGL entry has illegal length:address=0x34C9FFEC, length=0x14, cmd=X.
[28849.013916] 3w-9xxx: scsi1: ERROR: (0x03:0x0104): SGL entry has illegal length:address=0x34C9FFEC, length=0x14, cmd=X.

I don't know what it means but it looks bad..... and there are a lot of entries like that ones on the log.

[fzabkar]

Re: Is posible to tune HDD performance under linux?

All the measured transfer rates look low to me. However, it's not clear whether the buffered disk reads are random or targeted. A test that reads the data at the outermost tracks would guarantee the maximum result.

Here is a HD Tune read benchmark graph for the MQ01ABD075 (750GB) model (I couldn't find a graph for the 500GB drive):

http://www.notebookcheck.net/fileadm...10K/hdtune.PNG

The max transfer rate is 114MB/s at the outermost tracks and approximately 75MB/s at the innermost tracks.

The MQ01ABD075 has 4 heads whereas the MQ01ABD050 has 2. This means that the data density for the MQ01ABD050 is 250GB per head whereas the MQ01ABD075 has 188GB per head. Therefore the 500GB model should have a significantly faster transfer rate.

Toshiba Product Specification for MQ01ABD075 and MQ01ABD050:
https://cdn.badcaps-static.com/pdfs/...8fd8211182.pdf

The 2TB drive (/dev/sdd) is performing very poorly. A 1GB-per-platter 7200RPM drive should benchmark at around 180MB/s to 210MB/s maximum sustained read transfer rate.

ISTM that the HDDs are being seriously throttled by the motherboard or SATA cards. I suspect that if you run a HD Tune (or equivalent) read benchmark against them, you will see a long 60MB/s plateau at the start of each graph.

The "Invalid command opcode:opcode=0x85" is alerting you to a failure in the ATA PASS-THROUGH(16) command:

https://en.wikipedia.org/wiki/SCSI_command

ATA PASS-THROUGH is a SCSI command that enables any ATA command to be tunneled through to the drive by encapsulating it within a special SCSI packet. Software will typically use this method to send SMART commands to the drive (SMART is not supported directly by SCSI).

My Google searches would suggest that the other error ("SGL entry has illegal length") is a driver bug rather than a real problem with the HDDs. Several authoritative sources state that these errors can be ignored.

I would use dd to copy 1GB from LBA 0 to /dev/null. That should provide the best possible benchmark result.

[hikaruichijo]

Re: Is posible to tune HDD performance under linux?

I've found that ipfire tries to enable smart on all drivers at boot with smartmontools and the 3ware card doesn't support that, there's the cause of the errors.
Now I'm doing test with the kernel's read-ahead cache settings for the device.
(blockdev command and it's getting better. I've also found that this 3ware cards are very bad performers overall an triky to configure.
Another problem is that I don't have the BBU for the card and it disables the cache by default so I have to enable it using the 3ware tool on every boot, I have the server on an UPS so no worries about lossing data.
I'm using an script that I've found to check for the best readahead value for my drives, I'll post it later if it works...

[fzabkar]

Re: Is posible to tune HDD performance under linux?

The following threads suggest that you should be using a command line such as ...

smartctl -a -d 3ware,0 /dev/tw0

http://smartmontools-support.narkive...taii-harddisks
http://www.avestalc.com/questions/67...vices-in-3ware

You should be able to retrieve detailed information from:

sudo /sbin/hdparm -I /dev/sda

smartctl is also 3ware aware.

smartctl -a -d 3ware,0 /dev/tw0

The above will check disk health and info for 3Ware port 0 that's behind the presented RAID drive /dev/tw0. It might also be /dev/sd* or /dev/twe*.

If you have tw_cli installed, you can do tw_cli info for a list of controllers, then for example tw_cli info c0 to see controller 0. This will include the serial number of member drives.

[hikaruichijo]

Re: Is posible to tune HDD performance under linux?

Thanks for loking for the right command to access smart on the 3ware card, now I have to find a way to make it work with the webui, I think I'll can't do that because I'm not a programer....and yes, I have tw_cli instaled and I'm using it to enable the cache.
But now I'm testing the 2tb drive and it's showing a top speed of around 66 MB/s on buffered reads with a readahead cache of 1280 and it is strange because I've tested a disk connected to the Sil 3112a card and reached around 90MB/s. Maybe the 3ware card is crap...
I think there's a problem with the 3ware driver but I don't know how to change or update the driver in linux......