File Corruption Problem (RAID5)

[Shodan486]

Hello all after a long time!

I'd like to be brief and simple, so let's put it in:

I've recently built a copy of my server for my friend - dual PIIIs, 2GB RAM and 4x 1TB drives in RAID5 setup. Ain't got any dedicated software for the controller, which is PROMISE SATA 300 for PCI(64)-X, for data rebuild or something, nor the controller itself with its firmware does not provide such an option, thus I never cared for it and so did my friend...

But he somehow ''damaged'' his data by the following steps: He was copying stuff when he decided to test the RAID 5 reliability by ejecting one of the drives during the copy process - success, nothing discontinued. But the rebuild was quite problematic - he left Windows scandisk (XP) to do the job, which was catastrophic. Let's not care about the 700GB of corrupted image files and movies, but focus on the private data he damaged, mainly photos: the files CAN be seen in the directory structure, but when you open a photo, you'll see a picture with a ''layer'' of grey rectangle placed over it, in all of the photos - simply corrupted...

My question is - can it be restored? (but I guess I already know the answer).

[b700029]

Re: File Corruption Problem (RAID5)

Recommended viewing: http://www.youtube.com/watch?v=JacKwC5S5ds

[Mad_Professor]

Re: File Corruption Problem (RAID5)

Tell your friend he's an idiot.
Why did he cut and paste important data?

Anyways a raid 5 should be able to handle one drive drop out, so data should be intact on the other three via disk parity, unless windows botched it with scandisk, you may not be able to recover it.

RAID is complicated should be left to those who know how to administrate one.

[Shodan486]

Re: File Corruption Problem (RAID5)

Gotta say I'm partially to blame for, too. I assured him nothing should happen, as you said by yourself, but not even me nor him did now that the rebuild was not natively possible by the controller, we did not examined it before doing any serious stuff (I've seen that setup only once by a glimpse). So I feel responsible too - and as an idiot too (not taking any offense).

Still I wish there was some way to restore the data. Every file (image file, avi, mp3) is somehow corrupted and I'm asking why it has become like that. Well I guess you're right Mad Professor - one should understand the problematics of RAID array. Well I do, but it seems that the process of data recovery / rebuild is a bit more difficult than I expected, my bad. Anyway, I'd like some advice to what I should do. I'm a bit worried and so is the guy. Just FYI - nothing has been erased / written on this array since then, no rebuild procedure has been performed, only the winXP did some magic and the results are well mentioned above.

REALLY thanks for any help or hints.

[shovenose]

Re: File Corruption Problem (RAID5)

Windows system restore maybe?

[severach]

Re: File Corruption Problem (RAID5)

RAID5

Always a mistake. RAID5 sacrifices speed and reliability for space. As you just found out, space is the cheap part. Once you get it fixed change it to RAID 1.

[Mad_Professor]

Re: File Corruption Problem (RAID5)

Always a mistake. RAID5 sacrifices speed and reliability for space. As you just found out, space is the cheap part. Once you get it fixed change it to RAID 1.

Raid 5 is reliable, if it wasn't, no one would be using it. In the event of complete array collapse it maybe difficult to recover the data. Frankly he was using a cheap raid controller that does raid 5, as it turns out; it sucks rebuilding.

I use mdadm raid 5, I had one drive failure, drop a new drive in, add it to the raid 5 via mdadm, rebuilt the array, viola! No problems.

I think people are misinformed about RAID.
RAID is not a back up. Always keep at least two copies of data around.

I have three large independent disk for my raid 5 array set up to rsync once a week.

[Uranium-235]

Re: File Corruption Problem (RAID5)

the reliability of raid5 is somewhat dependant on the hardware and software. In large enterprise networks, getting a pure hardware raid5 solution is your best bet. In small businesses its best to go with a pure software solution (not fake raid, mdadm, windows server 2003 raid5...or hacked xp pro raid hack). This makes rebuilding somewhat easier as long as you can get your system to boot, as with fakeraid--which your promise sata controller might be. I don't mind fakeraid for raid 1, but 5...too risky

[severach]

Re: File Corruption Problem (RAID5)

I'm not interested in how much faster or more reliable one brand is over another. What I'm interested in is what I can do with the drives after the system goes fubar. RAID 5 is not reliable because you can't operate the drives separately or on a different controller. With RAID 1 you can. The only thing RAID 5 does is to get you 2 drives for the price of 3. RAID 1 gets you 1 drive for the price of 2. Back when drives were expensive there was some value in the extra space. Now the extra port costs more than the savings from the extra space. RAID 1 beats all over RAID 5 for speed, reliability, ease of management, recovery, and choice of controllers.

Softraid is nice because it is completely controller independent. That makes Linux upgrades easy. The only work in RAID 1 is spreading 1 write over 2 controllers so well written softraid drivers are almost as fast as hardware RAID cards.

With mdadm running RAID 5 you gain little but lose a lot of performance. RAID 5 is a bad choice for softraid and fakeraid controllers.

RAID (the redundant kind) eliminates loss from drive failure. It's another way of saying "don't put all your eggs in one basket." RAID 1 makes copies of all your eggs and stores them in another basket. RAID 5 splits all the eggs and places the halves in 2 separate baskets then fills the 3rd basket with a calculated egg from which any missing half can be rebuilt.

Now OP has eggnog. What a mess!

RAID 5, just say NO!

[Uranium-235]

Re: File Corruption Problem (RAID5)

I'm not interested in how much faster or more reliable one brand is over another. What I'm interested in is what I can do with the drives after the system goes fubar. RAID 5 is not reliable because you can't operate the drives separately or on a different controller. With RAID 1 you can. The only thing RAID 5 does is to get you 2 drives for the price of 3. RAID 1 gets you 1 drive for the price of 2. Back when drives were expensive there was some value in the extra space. Now the extra port costs more than the savings from the extra space. RAID 1 beats all over RAID 5 for speed, reliability, ease of management, recovery, and choice of controllers.

Softraid is nice because it is completely controller independent. That makes Linux upgrades easy. The only work in RAID 1 is spreading 1 write over 2 controllers so well written softraid drivers are almost as fast as hardware RAID cards.

With mdadm running RAID 5 you gain little but lose a lot of performance. RAID 5 is a bad choice for softraid and fakeraid controllers.

RAID (the redundant kind) eliminates loss from drive failure. It's another way of saying "don't put all your eggs in one basket." RAID 1 makes copies of all your eggs and stores them in another basket. RAID 5 splits all the eggs and places the halves in 2 separate baskets then fills the 3rd basket with a calculated egg from which any missing half can be rebuilt.

Now OP has eggnog. What a mess!

RAID 5, just say NO!

yeah, even in enterprise enviorments, just going with embedded (1+0) raid is a good performance boost with keeping a good amount of space and redundancy. Of course your argument is somewhat forefit cause drives now are expensive again, and in enterprise enviorments, its worse, has been cause SAS 15k rpm drives can run 1$/gb, in that case using a hardware controller is ok, but still raid 5 is...dangerous. Cause the more drives you have, the more chance of killing your whole array (rebuilding say drive 8 while drive 12 crapps out). RAid 6 gives you a little more redundance, sacrificing the space of one more drive, but gives you dual redundancy

your egg compairason isn't quite accurate. It would be like each drive storing a little bit of parity egg from the previous egg, not dumping all parity into a single drive....thats raid 4. Which I think would not be that bad an idea of it had more support, and support for nesing a raid 1 as a parity array, making it raid 14, which as far as I can tell does not exist, or exists out of the first page of a google search, implying nobody really tried to implement it, even experimentally. though I guess it could be done. maybe setting up raid 4, then on the parity drive specification, finding a SATA male/female cable, hooking that from the actual sata port to a seperate storage system running raid 1. hmm

[Mad_Professor]

Re: File Corruption Problem (RAID5)

yeah, even in enterprise enviorments, just going with embedded (1+0) raid is a good performance boost with keeping a good amount of space and redundancy. Of course your argument is somewhat forefit cause drives now are expensive again, and in enterprise enviorments, its worse, has been cause SAS 15k rpm drives can run 1$/gb, in that case using a hardware controller is ok, but still raid 5 is...dangerous. Cause the more drives you have, the more chance of killing your whole array (rebuilding say drive 8 while drive 12 crapps out). RAid 6 gives you a little more redundance, sacrificing the space of one more drive, but gives you dual redundancy

your egg compairason isn't quite accurate. It would be like each drive storing a little bit of parity egg from the previous egg, not dumping all parity into a single drive....thats raid 4. Which I think would not be that bad an idea of it had more support, and support for nesing a raid 1 as a parity array, making it raid 14, which as far as I can tell does not exist, or exists out of the first page of a google search, implying nobody really tried to implement it, even experimentally. though I guess it could be done. maybe setting up raid 4, then on the parity drive specification, finding a SATA male/female cable, hooking that from the actual sata port to a seperate storage system running raid 1. hmm

I just want to say...
I use mdadm raid 5 on six Samsung spinpoint F3 7200 1TB drives.

I guess I'm fucked aren't I?

[Uranium-235]

Re: File Corruption Problem (RAID5)

Might be. You purchased the drives at the same time? drives wear out at similar rates, if you purchased the same drive models at the same time. If one drive dies after say, 5 years, you have a much higher chance of another dying (which will destroy the whole array) while you're rebuilding, especially if you have 5, instead of say 2 to rebuild your data, also cause the face you're rebuilding A LOT more data

you currently have what? 5gb of storage

if you setup it in raid 10, you would have a maximum of 3, and decrease the chance of total array failure by like, 50%, without the decrease in performance

no biggie before the hard drive price hike, you would require 10 hds to have the same space, and have serious performance

[Mad_Professor]

Re: File Corruption Problem (RAID5)

Might be. You purchased the drives at the same time? drives wear out at similar rates, if you purchased the same drive models at the same time. If one drive dies after say, 5 years, you have a much higher chance of another dying (which will destroy the whole array) while you're rebuilding, especially if you have 5, instead of say 2 to rebuild your data, also cause the face you're rebuilding A LOT more data

you currently have what? 5gb of storage

if you setup it in raid 10, you would have a maximum of 3, and decrease the chance of total array failure by like, 50%, without the decrease in performance

no biggie before the hard drive price hike, you would require 10 hds to have the same space, and have serious performance

I got 4.5TB usable but I only have 1.7TB avail as of right now.

One of the reasons why I didn't go with raid 10 is space to price ratio. I already had 1.5TB raid 5 at 75% full. Instead of doubling it, I tripled it. It has lasted me while. I spent $500 on drives, another $200 on a PCI-E intel rebranded LSI card for it's two port 8087 with two 8087 to 4 sata cables. Even the card only had 0,1, 0+1, and 10; no raid 5. LOL. I used mdadm, since I was already familiar with it. With filebench I get 290Mb/s read 220MB/s write, fast enough for samba and my gigabit network.

I also have samsung 2TB and two 1.5TB seagates operating individually with rsync and separate data shares on the raid 5 for each device, once in awhile I would have to move stuff around to balance out the usage to keep them around 50%

I also have two WD RE4 500GB operating in mdadm RAID 1 for KVM guests.

All powered on a celeron dual core at 2.3ghz with 8GB of ddr2 memory plugged into a APC 1500VA battery backup.

Frankly I'm happy with my server.

[Uranium-235]

Re: File Corruption Problem (RAID5)

i'm sure, just have that spare drive ready and waiting if you do get failure, the more time without replacing it, the more chance of loosing the whole array