mass raid storage

[sidewinder]

Well for what I am doing this kind of needs its own thread.
I've been using raid for years. After having extremely bad luck for drive failures starting in 1994, RAID 1 or 5 is the only thing I will run nowadays.
Our main storage system is running raid 5 on this card (http://www.adaptec.com/en-US/support/raid/scsi_raid/ASR-3410S/) and with my luck, of course, I have lost one device. After silencing the internal alarm, replacing the failed drive was really a pleasure.

I'm starting to grow weary of having lots of drives though, so the time has come to put together a massive, dedicated storage system that will be available via 1 gig ethernet, and SATA raid appears to be the way to go as it has blown u320 and SAS out of the water for space. I have learned recently that creating raid arrays on certain motherboard chipsets can present migration problems when the time to upgrade occurs, so a dedicated SATA controller makes more sense. I was hoping to have a low budget for this project but it may not be possible. While speed is a non-issue, redundancy and maximum storage are.
Configuration thoughts include:

Case - Microcool Banchetto for now
Qty 4 to 9, 1 to 2 terrabyte 3.5" drives. The banchetto could hold up to 9 with an appropriate hot swap internal enclosure.
Board, memory and cpu doesn't really matter as it will be just serving up data via ethernet. It may even run linux for stability and uptime, but that will all depend on the controller. Of course, it will be liquid cooled. :)

Possible controller cards: AMCC/3Ware 9650SE-12ML or Areca ARC-1220
This is where I am looking for some advice. If anyone has knowledge with either of these or other controllers, capable of handling 8 or more drives in RAID 0/1/5/6 modes I'd like to hear from you. Nothing is set in stone yet so any alternatives would be great. TIA - Gary

[Cutless009]

Just personal opinions here mind you, but put together from my years of experience with hard drive crashes (I seem to have the same luck as you :-/)

Hard Drives:

I would go with these hard drives if new egg ever gets them back in stock, they are as fast as the spinpoint F1's from samsung, but will last longer due to less stress on the internals thanks to WD's new technology (inteli-seek), plus its dual processor, how awesome is that lol xD:

http://www.newegg.com/Product/Product.aspx?Item=N82E16822136284



Raid Card:

You can never go wrong with Areca, and for what you're doing Id reccommend this card:
http://www.newegg.com/Product/Product.aspx?Item=N82E16816151005

it has 16 3gbps sata II ports, and an upgradeable onboard SO-DIMM ECC memory chip (upgradeable to 1gb onboard ram).

If you're looking for something a little lower key/budget happy, you could always go with this one:
http://www.newegg.com/Product/Product.aspx?Item=N82E16816131004

but you lose the upgradeable ram, and it only has half the ports on it, so if you ever wanted to upgrade to more drives later you wouldnt be able to.

[sidewinder]

Thanks for the reply Cutless. Well, I have had one or two WD's fail, and one of my local comrads just had a 500 gig mybook die this weekend. I have had good luck with the Hitachi 1 tb drives as of late, I believe they were among the first out with their 7k model (knock on wood)
Whatever drive we end up going with it definitely won't be oem with a 1 year warranty. I'd prefer SAS myself but the price per gig doesn't cut the mustard. I saw that 1260 controller as well, there were more complaints on the 1220. The 1260 has a mass of ports which is nice for upgradeability, no need to throw away old drives. I would not mind a multi-lane controller either, just a bit tough to blow that kind of cash on just the board but.. Nvidia raid is not to be trusted even when migrating to another nvidia board. I also don't think Nvidia has this feature either unless its been implemented in their newer chipsets:

Online hot swap: Automatic drive insertion/removal detection and rebuilding

So, I think I have the controller down now at least.

[Septim]

shocking prices for a good dedicated Raid Controller...

WD 1TB caviar black with 32mb cache and 2 procies should do you proud...

knock on wood on hard drive crashes +1...

[Cutless009]

just a heads up, WD cheap drives suck, and the ones they put in the mybooks arent even their caviar series, they are their most generic drives. Even good companies can have bad products *need I said 9800 GX2 or Nvidia Chipset?*

[Hondacity]

westerndigital offers two versions...

1tb - caviar black
1tb - re3

re3 is their server class hd

http://www.wdc.com/en/products/Products.asp?DriveID=503
http://www.newegg.com/Product/Product.aspx?Item=N82E16822136313&Tpk=WD1002FBYS

[Cutless009]

I can get the 500gb version of those RE3's for $50 all day long :) thats what I run in all my computers. I wonder if my supplier has any 1tb's.

I can personally attest to the fact that they are quiet, quick, and stable, but they do tend to run a bit hot.

[Septim]

This Caviar Black (WD1001FALS) Fast and Furious?...
http://www.wdc.com/en/products/products.asp?DriveID=488

$120 at http://www.newegg.com/Product/Product.aspx?Item=N82E16822136284&Tpk=WD1001FALS

[Kayin]

I'm building an array almost just like this at home, and I'm using 4xSamsung Spinpoint F1 (Because I will not willingly let a WD in my house, I've had every one I've owned fail on me) and http://www.newegg.com/Product/Product.aspx?Item=N82E16816115027 this for a controller. Granted, mine isn't QUITE as critical, but I do host a lot of stuff at home: artwork, SketchUp files, etc. that I don't wanna lose. Also, I store system backups of everything, so this will help me free up a lot of space as well...

[Cutless009]

I dont trust highpoint, theyre like the durex of raid controllers, u never know when their gonna break, and then ur shit outta luck :down:

[Kayin]

Well, then I gotta go up to a controller twice the price, and this is already getting to be too expensive a venture. I may just buy a few drives and be done.

[rubidium]

I have lots of personal and business-related experience with Areca's ARC-1120, the PCI-X brother to the 1220 you are considering. They are fast and reliable. My one at home has been running flawlessly in RAID5 24/7 for about 3-1/2 years now. At work, we use them in data collection rigs where we need to write a 5-second long burst of high sample rate data from an A-to-D converter as fast as possible before repeating the same. The card even runs well in some extreme cold/hot outdoor temperature environments without a hitch.

I also have to admire Areca's customer service. When I first bought the 1120 for my home build using a Tyan S2895 motherboard and 150GB Raptors, there was apparently a wierd race condition created jointly by timing issues associated with the chipset and the HDD's, that caused the 1120 to hang indefinitely when booting. I emailed Areca about the problem, and within a few hours received a response from an engineer in Taiwan, outlining a simple test procedure he requested I perform. Within a few hours of my emailing back of the test results, there was a frenzy of back-and-forth email traffic among Areca, Tyan, and WD engineers that I was cc'd on. Among them was an urgent request to WD to overnight ship 8 Raptors to Taiwan, since they didn't have enough of them handy. Two days later, the Areca engineer that originally contacted me sent me a beta BIOS for my 1120 and asked me to test it. It worked like a champ. Talk about moving heaven and earth for the little guy.

Some years back, I made up an Excel spreadsheet that performs all the binomial theorem calculations to estimate probabilities of array failure for different RAID configurations, given the MTBF of a single drive. I can send it to you if you like, or even tweak it to represent your own personal configuration. A few snapshots for an MTBF of 1.2 million hours are:

8 Drives in RAID0
Years Continuous Running / Probability of Single HDD Failure / Probability of Array Failure
1 / 5.5% / 5.7%
2 / 10.5% / 11.0%
3 / 14.9% / 16.1%
4 / 18.8% / 20.8%
5 / 22.2% / 25.3%
6 / 25.2% / 29.6%
7 / 27.9% / 33.6%
8 / 30.2% / 37.3%
9 / 32.1% / 40.9%
10 / 33.8% / 44.3%

8 Drives in RAID5
Years Continuous Running / Probability of Single HDD Failure / Probability of Array Failure
1 / 5.5% / 0.1%
2 / 10.5% / 0.6%
3 / 14.9% / 1.2%
4 / 18.8% / 2.1%
5 / 22.2% / 3.1%
6 / 25.2% / 4.3%
7 / 27.9% / 5.7%
8 / 30.2% / 7.2%
9 / 32.1% / 8.8%
10 / 33.8% / 10.5%

8 Drives in RAID6
Years Continuous Running / Probability of Single HDD Failure / Probability of Array Failure
1 / 5.5% / 0.002%
2 / 10.5% / 0.02%
3 / 14.9% / 0.05%
4 / 18.8% / 0.1%
5 / 22.2% / 0.2%
6 / 25.2% / 0.4%
7 / 27.9% / 0.6%
8 / 30.2% / 0.8%
9 / 32.1% / 1.1%
10 / 33.8% / 1.5%
rubidium

[Cutless009]

if ur on a budget, why not just go with a NAS box:

http://www.newegg.com/Product/Product.aspx?Item=N82E16822102016

[Kayin]

That's more than I was gonna spend anyway.

I'm just gonna shove some more drives in there and burn more backup discs. Keep throwing money at diminishing returns.

[Septim]

i see lots of DVD DL in my future and 2 DVD writers with more than 300 successful writes each...

[sidewinder]

I dont trust highpoint, theyre like the durex of raid controllers, u never know when their gonna break, and then ur shit outta luck :down:

I have read the same regarding highpoint

[sidewinder]

I have lots of personal and business-related experience with Areca's ARC-1120, the PCI-X brother to the 1220 you are considering. They are fast and reliable. My one at home has been running flawlessly in RAID5 24/7 for about 3-1/2 years now. At work, we use them in data collection rigs where we need to write a 5-second long burst of high sample rate data from an A-to-D converter as fast as possible before repeating the same. The card even runs well in some extreme cold/hot outdoor temperature environments without a hitch.

I also have to admire Areca's customer service. When I first bought the 1120 for my home build using a Tyan S2895 motherboard and 150GB Raptors, there was apparently a wierd race condition created jointly by timing issues associated with the chipset and the HDD's, that caused the 1120 to hang indefinitely when booting. I emailed Areca about the problem, and within a few hours received a response from an engineer in Taiwan, outlining a simple test procedure he requested I perform. Within a few hours of my emailing back of the test results, there was a frenzy of back-and-forth email traffic among Areca, Tyan, and WD engineers that I was cc'd on. Among them was an urgent request to WD to overnight ship 8 Raptors to Taiwan, since they didn't have enough of them handy. Two days later, the Areca engineer that originally contacted me sent me a beta BIOS for my 1120 and asked me to test it. It worked like a champ. Talk about moving heaven and earth for the little guy.

Some years back, I made up an Excel spreadsheet that performs all the binomial theorem calculations to estimate probabilities of array failure for different RAID configurations, given the MTBF of a single drive. I can send it to you if you like, or even tweak it to represent your own personal configuration. A few snapshots for an MTBF of 1.2 million hours are:

8 Drives in RAID0
Years Continuous Running / Probability of Single HDD Failure / Probability of Array Failure
1 / 5.5% / 5.7%
2 / 10.5% / 11.0%
3 / 14.9% / 16.1%
4 / 18.8% / 20.8%
5 / 22.2% / 25.3%
6 / 25.2% / 29.6%
7 / 27.9% / 33.6%
8 / 30.2% / 37.3%
9 / 32.1% / 40.9%
10 / 33.8% / 44.3%

8 Drives in RAID5
Years Continuous Running / Probability of Single HDD Failure / Probability of Array Failure
1 / 5.5% / 0.1%
2 / 10.5% / 0.6%
3 / 14.9% / 1.2%
4 / 18.8% / 2.1%
5 / 22.2% / 3.1%
6 / 25.2% / 4.3%
7 / 27.9% / 5.7%
8 / 30.2% / 7.2%
9 / 32.1% / 8.8%
10 / 33.8% / 10.5%

8 Drives in RAID6
Years Continuous Running / Probability of Single HDD Failure / Probability of Array Failure
1 / 5.5% / 0.002%
2 / 10.5% / 0.02%
3 / 14.9% / 0.05%
4 / 18.8% / 0.1%
5 / 22.2% / 0.2%
6 / 25.2% / 0.4%
7 / 27.9% / 0.6%
8 / 30.2% / 0.8%
9 / 32.1% / 1.1%
10 / 33.8% / 1.5%
rubidium


That is also part of the reason I was thinking the extra dollar towards raid 6 technology was worth it. Those are ridiculously low %'s and I will take those. Thanks Rubidium!

[sidewinder]

Just a quick update, this may turn into a sort of build log, and perhaps a review of the controller card. Here is what we have coming to build these two systems thus far:

14 pcs 1 terrabyte Samsung HD103UJ - oem (3 yr warranty)
2 pcs Areca ARC-1220 8-port controller card
2 pcs Norco RPC-4020 R

I have several 1 tb hitachis that will bring the total quantity to 16. Each enclosure will hold 8 drives to start. I would have liked to have used the banchetto for this, but after much consideration the banchetto's specialty is being a test bench. And these parts are not to be tested and poked. We need something that is hot swappable, plug it in, turn it on and leave it. And the Norco seems to have it. Its going to get a bit of a boost in the cooling department though for sure. I'm anxious to see what kind of transfer rates they'll be putting out. I figure by the time I need additional storage, I can deal with the issue of the controller and/or second array bundle.

[rubidium]

We'll be interested in seeing the results. You'll inevitably want to do some experimentation with alternative stripe sizes and block sizes to optimize your performance for your application(s) - the speed vs efficiency trade.

rubidium

[Cutless009]

nice man that norco case is awesome. I wish I had the money to set something up like this in my house, we shoot tons of video around here (mostly of my little girl lol) and I do digital arts and photography, so I have thousands of gigs of crap lying around that I dont have space to back up, and if I want to share it I have to send it over msn, GHEY.

[Fitseries3]

if it helps you any...

theres a guy on OCN selling dell perc 5/i's from brand new servers. he's selling them for $85 which is a steal.

you can flash them to a LSI MegaRAID SAS 8480E and get even better performance.

they are a sas card thats pcie 8x. theres a small issue that has to be taken care of before using it on a non dell board but its VERY simple.

all this info can be found here.... http://www.overclock.net/hard-drives-storage/359025-perc-5-i-raid-card-tips.html

totally kick ass card!

[Cutless009]

wow thats a MASSIVE steal man.