Call to anyone with RAID 1E experience

[PCBONEZ]

Re: Call to anyone with RAID 1E experience

actually, for once, you're wrong, by 8 (factor of)

122-204 MB/s

one site has it average 166 MB/s = 1328 Mb/sec = 1.3Gb/s, almost to 1.5 gb, but it bursted to 200 MB, which is past 1.5

You're right.

And the latest Velociraptors are at 145 MB/s so I hope that extra 21MB/s is worth the ~$100 per drive you're paying for it.
.

BTW: If you have a server with a single Gbit NIC, drives over 125MB/s will do you no good anyway.
.

[Digital Technophile]

Re: Call to anyone with RAID 1E experience

I had a flaky 120GB IDE hard drive that would corrupt data without the drive or controller noting any problem. I diagnosed it by testing copied ZIP files. When I got a SMD tool I replaced the RAM chip and the drive works fine.

I looked high and low for a disk RAID level that would use ECC to help ferret out failing drives that don't know they are failing. Raid 5 and 6 aren't it and none of the others are either. The industry seems unconcerned that drives can be something other than perfect or non working.

Raid 5 buys space at the expense of convenience and speed and Raid 6 uses extra drives to keep the speed up with failed drives. Parity was clever back when hard disks were more expensive than card processors. It's counter productive now that hard drives are so cheap. Once you give up on Raid 5 as a relic from the past a lot of el-cheapo RAID cards without hardware XOR become practical and you can spend the money you'd waste on a phat RAID card on more drives.

Raid 1 Mirror FTW!

I know why your 120GB didn't note any problems. A lot of people don't know that SMART drive error reporting is built on a little known technology called S.T.U.P.I.D. (Supposed To Utilize Program Integrity Doesn't). Its purpose is to make SMART as useless as possible. How useful is a monitoring software that can't read a disk's SMART data because it's connected through RAID array for instance?? Or that parameters are black and white devoid of a meaningful interpretation of its values other than "threshold" reached. SSD manufacturers need to really study this bit of storage history so that their "wear-leveling" algorithm can alert a user that the secret reserve space is becoming near full capacity and perhaps should think about replacing SSD. Think I'll keep my expectations low and hope to be pleasantly surprised if such a system is actually implemented.

I appreciate your insight into RAID 5 & 6, has it become that irrelevant? Curious why HBA manufacturers keep these around? I wholeheartedly agree that RAID-1 is best all purpose mode to protect from drive faults. But what's the point of mirroring corrupt data? Was your flaky 120GB part of a RAID-1 array? Or did backups play a role in data protection? Or did you lose data?

I suppose I really need to get my hands dirty on this one. I would like to investigate first hand RAID 5 & 6, but for now as stated earlier my soon to be reinstated domain controller will have RAID-1. And for sanity's sake, the rig I'm planning for VMware ESX will have RAID 1+0 as its primary storage. When the time comes to retire my current socket 939 board, it might be repurposed into a RAID lab.

And PCBONEZ and Uranium....

forget APES and ORANGES....

we're talkin' POTATOES and TOMATOES!!!

[PCBONEZ]

Re: Call to anyone with RAID 1E experience

If the array is mirroring corrupt data that means it's writing which means it's receiving corrupt data from somewhere else.
Somewhere else is probably before the controller card.
If the array [card] has corrupted data coming in to start with then how is some parity function [initiated in the RAID card] going to know that the data is corrupt?
- I don't think it will...
.

[Uranium-235]

Re: Call to anyone with RAID 1E experience

Raid 10 FTW. more size, more speed, redundancy

[Digital Technophile]

Re: Call to anyone with RAID 1E experience

If the array is mirroring corrupt data that means it's writing which means it's receiving corrupt data from somewhere else.
Somewhere else is probably before the controller card.
If the array [card] has corrupted data coming in to start with then how is some parity function [initiated in the RAID card] going to know that the data is corrupt?
- I don't think it will...
.

Excellent point! Wouldn't it stand to reason though if disk 1 experienced a bad sector, then it would be mirrored to disk 2 (RAID-1)?

[PCBONEZ]

Re: Call to anyone with RAID 1E experience

No, because what ever was written to the sector that became bad was already written to disk 2 when it was initially written to disk 1.
The card should know which has the bad sector, remap, and restore it from disk 2.

[Digital Technophile]

Re: Call to anyone with RAID 1E experience

No, because what ever was written to the sector that became bad was already written to disk 2 when it was initially written to disk 1.
The card should know which has the bad sector, remap, and restore it from disk 2.

Alright, I can work with that, makes perfect sense. Thank you for pointing that out.

As far as RAID-1+0 goes, nothing other than available ports are stopping me from running as many even number of drives (thinking of a 6 drive array=3+3 mirrored), correct?

[PCBONEZ]

Re: Call to anyone with RAID 1E experience

Not sure where you're going.
3+3 mirrored is RAID-1 not 10.

I do mine like this:

Array-1 [OS]
1+1 drives mirrored

Array-2 [Data]
(however many) drives mirrored

+ Hot Spare(s)
If a drive fails the array creates a new mirror on the Hot Spare from the good drive automatically.
It might be finished before you even know you have a bad drive.
.

[PCBONEZ]

Re: Call to anyone with RAID 1E experience

Here's an upgrade trick for you.
-
[[ Not sure if it works in XP but I've done it several times in W2k.
[[ As you don't change the controller I 'THINK' it would work in XP just fine.

Starting point is a mirror with smallish drives. [I'll say 40Gb.]
Goal is to install bigger drives. [250Gb.]

Remove any existing hot spare.
Install one 250Gb as a hot spare.
Unplug [fake fail] one of the 40Gb drives and let the mirror rebuild onto the 250Gb.
Install the second 250Gb as a hot spare.
Unplug the second 40Gb.
Let the mirror rebuild onto the second 250Gb from the first 250Gb.
Now the files are on the two 250Gb and mirrored but only 40Gb is being utilized.
Resize the partitions [or create new] to use all 250Gb. - Done.
-
And you never wipe the 40Gb drives that way so if something screws up you can put them back and start over as long as you put them both back at once so they don't get rebuilt.
.

[Digital Technophile]

Re: Call to anyone with RAID 1E experience

Not sure where you're going.
3+3 mirrored is RAID-1 not 10.

I do mine like this:

Array-1 [OS]
1+1 drives mirrored

Array-2 [Data]
(however many) drives mirrored

+ Hot Spare(s)
If a drive fails the array creates a new mirror on the Hot Spare from the good drive automatically.
It might be finished before you even know you have a bad drive.
.

Sorry PCBONEZ, it was getting late so I wasn't clear about that. What I was trying to say was 3 striped + 3 mirrored, but I see that is possible. From what I understand, you can have as many even number of drive the card will support.

I did want to ask about that 3ware 9500. How is it possible to run that in a PCI slot if it's PCI-X?? What do those extra pins hanging out (covered with electrical tape) do anyway?? I don't see this card offered new (maybe you know where), but I see a bunch on fleabay. Any thoughts?

[PCBONEZ]

Re: Call to anyone with RAID 1E experience

http://computers.shop.ebay.com/Compu...=p3286.c0.m282
-
I have quite a few of them from 4 to 12 ports.
Scoring a 12 port for $<100 is not impossible.

How is it possible to run that in a PCI slot if it's PCI-X

Because the PCI-X specification it's built to is the industry standard which requires it to be backwards compatible with 32-bit PCI slots.
[The 9550 is built to a different PCI-X spec which drops the +5v option and that makes it incompatible. There is also a 'key' in the slots which is different between 3.3v and 5v and that ensures you don't put a 3.3v card in a 5v slot.]

The extra pins are 'data only' for the other 32-bits in a 64-bit application.
Same reason/way/scenario as how/why PCI-E 8x is longer than PCI-E 4x but an 8x card works in an 4x slots.
The extension is 'data only' making the cards interchangeable in different length slots but at different bandwidths.
- That's a really old technique that dates back (at least) to increasing the 16-bit bandwidth of ISA slots with a 32-bit VLB extention.
.

[Per Hansson]

Re: Call to anyone with RAID 1E experience

I think you should carefully consider your requirements.
I really don't agree with the logic of using small cheap drives as discussed in RAID arrays

Why use drives costing mere bucks or given for free together with $500 (if entry level) hardware RAID controllers?
Performance will suck and so will reliability.

If performance is what you are after then I'd just skip the hardware RAID & get 2x of the Intel 320 series SSD's and run those in RAID-1 (They come in capacities up to 600GB each)
Or just use 1x SSD and use backup software to mirror it via network to a NAS or something, much better reliability (no single system point of failure)
Remember that dataloss is not just to disk failure, it can be user error, data corruption due to software issue
Theft, Fire, Flooding

The list goes on, RAID will not help with any of this...

[Digital Technophile]

Re: Call to anyone with RAID 1E experience

I think you should carefully consider your requirements.
I really don't agree with the logic of using small cheap drives as discussed in RAID arrays

Why use drives costing mere bucks or given for free together with $500 (if entry level) hardware RAID controllers?
Performance will suck and so will reliability.

If performance is what you are after then I'd just skip the hardware RAID & get 2x of the Intel 320 series SSD's and run those in RAID-1 (They come in capacities up to 600GB each)
Or just use 1x SSD and use backup software to mirror it via network to a NAS or something, much better reliability (no single system point of failure)
Remember that dataloss is not just to disk failure, it can be user error, data corruption due to software issue
Theft, Fire, Flooding

The list goes on, RAID will not help with any of this...

Thanks Per,

Well I have very carefully considered my requirements. This thread is becoming quite long now and if you have been following will have realized that there are two systems I'm working on. The near term system is an Intel D850EMV board I have that needs a little cap love, a RAID controller, and some PC1066 RDRAM. I've pretty much settled on a controller (board has PCI 2.2 5v slots/AGP...no PCIe) and probably going to use some 160GB SATA II drives I already have in lieu of buying some inexpensive WD Raptors (money factors). Its purpose will be a domain controller/DNS server in a small network, so I may use it for other minor purposes. But RAID-1 is first choice for primary storage, possibly (but not likely) RAID-1+0.

Now the big rig which this thread was was in consideration of will likely start out with a PCIe-based RAID controller/Mech. HDD setup, with dreams of PCIe SSD storage in the long term. As far as SSDs go, I think that (1) 2.5" FF SSD are bit silly (unless you are retrofitting into older system), (2) PCIe-based SSD are better throughput (PCIe 2.0 x4 = 16 Gbit/s [2 Gbytes/s] , PCIe 2.0 x8 = 32 Gbit/s compared to SATA II = 2,400 Mbit/s [300 Mbytes/s] , SATA III = 4,800 Mbit/s), AND (3) SSD flash memory (IMHO) still haven't had the revolutionary leap in MTBF reliability/ price point necessary to implement as primary storage medium. As it stands now, I would consider only using them as ROM-like devices, writing to them sparingly as possible to preserve longevity. And Intel may build some hard rock CPUs, ICs, and mainboards, but reading about their SSD firmware issues doesn't inspire confidence in their offerings at this time. I imagine though they will eventually rise to prominence in this sector. But I digress.

Big thanks to PCBONEZ, Uranium-235, and everyone who's helped me sort out these RAID modes (I may still "play" with RAID 5 & 6 on a small, "non-mission critical" scale for tickles and grins and to satisfy my own curiosity...I like fire!!!!)

[PCBONEZ]

Re: Call to anyone with RAID 1E experience

What are you using the 'big rig' for?
-
One of the basic points I've been trying to get across is that the drive interface speed is -rarely- where the bottle necks are.
-
The media transfer rate [head/disk] is often the limiting factor.
-
If you are sending data through a 32-bit PCI BUS the PCI BUS may be the limiter.
-
If sending files through a LAN the LAN speed may be a limiter.
-

Analyze the whole path from the hard drive's platters to the RAM in the machine that will use the files.
- You may find that faster at one point in the path is irrelevant due to limitations at another.
Knowing that can save you some $ by not buying gear that won't do you any good anyway.
.

[Digital Technophile]

Re: Call to anyone with RAID 1E experience

The big rig (still in planning) I'd like to try running VMware ESX Server on. That's what all the to do over RAID-1e,5,6, AND 1+0 has been about. I think I'm pretty well settled on a RAID 1+0 6 drive array (2x 3 way stripes, mirrored); that should give the PCIe bus something to chew on. The next thing is to find out what ESX server supports for hardware because I've been looking at 6-core AMD Phenoms and that MSI 890FXA-GD70 mainboard; but ESX requirements will ultimately dictate what that machine will be. Just for once I'd like to buy something new to put together. BUT, after having been burned in the 1TB SATAIII debacle, I will be scrutinizing every bit of hardware for that build.

AND the other machine the Intel D850EMV is what I'll be running RAID-1 on Windows 2003 (3ware 9500 we spoke of earlier; pretty good prices on fleabay) on a 32-bit PCI bus, so I'm really not all too worried about speed (can't remember for sure, but I'm hoping for 66mHz bus speed...manual and tech docs not too clear other than 5v PCI 2.2). either way 133Mbytes/sec or 266Mbytes/sec fine for what this machine will be doing anyway.

Sorry, I sometimes get long winded as I tend to over explain things. Which devolves into boring reading I'm sure (are you still awake?). But it's funny how a simple question (or was it?) about that new RAID mode I heard about turned into a gigantic thread!!! It's like "Why's the sky blue?" turning into "Well what kind of black hole do you think will devour a pulsar.." type conversation.

Maybe my sig should just say:

...BUT I digress

[shadow]

Re: Call to anyone with RAID 1E experience

It has been a long time since I have been able to go on this site!

I remember some time ago there was an interesting read here about the pro's and con's of hardware RAID controllers. I am unsure about the purpose of building this server. However it should probably be considered.

The primary issue with hardware RAID controllers is what do you do when the card itself fails? Quite often you are forced to use exactly the same card again. This may not be feasible in the future (assuming it is hard to source). Some RAID cards will also require the same firmware to be flashed on the card itself, otherwise it will not recover.

Having said this, I still like the idea of a hardware RAID controller!
I love the concept of having a piece of hardware do all the hard work so that the main CPU and RAM can be used for more productive things. On top of that, I have a strong suspicion that the performance on a hardware based version would be superior to a software based solution.

[severach]

Re: Call to anyone with RAID 1E experience

The primary issue with hardware RAID controllers is what do you do when the card itself fails? Quite often you are forced to use exactly the same card again. This may not be feasible in the future (assuming it is hard to source). Some RAID cards will also require the same firmware to be flashed on the card itself, otherwise it will not recover.

Raid 1 FTW! So long as the card doesn't require a 1 sector shift and there's no Raid 0 the drives will run on a non RAID controller until you can come up with another RAID solution. Any RAID controller that can promote drives to RAID without data loss won't implement the 1 sector shift.

There's no calculation and no hard work with Raid 1. It's two writes for every one. Hardware RAID shouldn't be very much faster than software RAID. The main advantage of hardware RAID is the battery backed write cache in case of sudden power failure. I decided that the benchmarking talking heads were wrong and that writing doesn't need to be fast so I opted for softRAID 1. It works quite fine.

The WD 120GB IDE drive that had a bad memory chip wasn't in a RAID system. It was running in a gaming rig and storing CD images. The bit errors were so rare that the system was mostly functional. It took a while to pick the drive out of the many suspects and a while longer to figure out how it should be tested.

That seems to be a very painful way to increase a 40GB RAID to a 250GB RAID. Ghost seems so much easier. Besides, it looks to me like it wouldn't work. You must specify the array size during creation. On the 250GB drives the array size would still be 40GB so only 40GB would be visible to the OS. Given the borderline incompetent options provided in RAID software and BIOS I'd be surprised if any brand allow arrays to be enlarged.

[PCBONEZ]

Re: Call to anyone with RAID 1E experience

That seems to be a very painful way to increase a 40GB RAID to a 250GB RAID. Ghost seems so much easier. Besides, it looks to me like it wouldn't work. You must specify the array size during creation. On the 250GB drives the array size would still be 40GB so only 40GB would be visible to the OS. Given the borderline incompetent options provided in RAID software and BIOS I'd be surprised if any brand allow arrays to be enlarged.

Works fine.
As far as I'm concerned it much less painful than messing with Ghost.

""On the 250GB drives the array size would still be 40GB""
No kidding! - Mirrors are the SAME SIZE!! - Imagine that!!!!
And now read the rest of it. - The last line.

Resize the partitions [or create new] to use all 250Gb. - Done.

[severach]

Re: Call to anyone with RAID 1E experience

And now read the rest of it. - The last line.

You didn't read what I wrote. Vista, Windows 7, and many disk utilities are more than happy to enlarge partitions so long as the disk controller exposes it. Even after both 250GB drives are in place the array size will still be 40GB. The space won't be exposed to the OS and enlarging will not be possible. No RAID BIOS I've seen allows altering the array size without losing data. Most don't allow much of anything in the way of changes without losing data.

[PCBONEZ]

Re: Call to anyone with RAID 1E experience

I suppose that's a problem for those people that use Win7 or [heaven forbid] Vista but since I don't and don't intend to any time soon it's a non-issue.

Yes, trying to change the array size through the RAID BIOS would do that.
That would be called "reconfiguring the array" - which is NOT what I said to do.
In fact it would be stupid to even TRY that.
Which is why I didn't get what you did. - Why did you try that? You should have known it wouldn't work.
-
If you follow my little routine then you NEVER have to reconfigure the array at all and the down time is reduced to what it takes to swap drives. [The system can be in full use during the mirror rebuilding. - Very little actual down time.]

-
What I DID say was to "resize the partitions", which is done through the OS using one of the many utilities available for that purpose.
While I obviously haven't tried, I'm sure they work well in Vista and Win7 too, at least better than using the trash MS utility included with the OS.
.