Ratdude's 3U Server Shenanigans

[ratdude747]

If some of you have seen, Topcat has a lot of 2TB WD green's floating around.

Today I found out what they came out of, and apparently I get to trade a less than useful IBM x3650 (takes a 4 post rack, I have a 2 post), two ancient 1u Supermicro blades,an AFT XM-5u card reader with 5.25" adapter, and some laughs at my expense for it.

And what is it? He didn't say, but based on some grainy cell phone pics, it is a Rackable Systems S3116! 16 SATA hot swap bays with 2 internal bays.

See attached manuals. It's a "custom" server made by SGI (who uses the "Rackable systems" brand) that has a Tyan S2882 Thunder K8S Pro motherboard in it.

The bad news is it only has one CPU and cooler. The good news is I still have a pair of 2.6GHz Opteron X2's leftover from V3.5's death and revival and also a pair of dynatron coolers (which matches the one installed). I even have 8GB of DDR-400 that should max it out (that is, with the DDR running the full 400 and not 266). It's a server, yes, but I have the hardware, so why not?

It's the server version of the K8W's I've had all sorts of fun with, but with no AGP controller to give me hell, I'm set to jet.

The best part is it appears to be two-post compatible out of the box, so it's a great fit.

It will be missing the HDDs and SAS cables, but hopefully I can talk Topcat into leaving the 3Ware 9550 card in it as my spare 9550 only has 4 ports...

Unless Work calls, looks like a weekend trip is coming in a few days...

[Topcat]

Re: Ratdude's 3U Server Shenanigans

Ok, the background on this contraption....it was given to me with the twin motherboard server case in THIS THREAD.

It rode under the big roll-around server on a freight truck from Georgia:



I even tried to give it away once, nobody stepped up:
https://www.badcaps.net/forum/showpo...7&postcount=14

It had about a dozen 2TB WD greens in it. Amazingly enough, they were all good. I cleaned the contacts on the interface boards and sold a few off cheap, I still have about 7 or 8 of them, which I'll be keeping.

Ok, so fast forward to yesterday. Ratdude called and asked some questions about some unrelated stuff, and we got on the subject of his rack project and NAS's....and I remembered I had this thing....and the rest you can figure out.

Anyway, I robbed the mini-sas cables out of it, I need them for my twin motherboard NAS project (they're 40" long)... I also removed all the drives and destroyed all the data (as per my agreement for accepting this machine). Beyond that, I've done nothing with this machine....till today. I figured I'd make it kinda/sorta presentable if someone was going to drive all the way from Indiana to come get it. I took it outside and gave it a blast with the air compressor hose....a big brown cloud emerged. I also noticed the RAM was configured wrong for this board only having one processor, so I corrected that....and then noticed that the southbridge heatsink was missing a peg. I found the peg in the bottom of the case, but the spring is MIA....so that'll be something for Ratdude to remedy.



The missing Peg:


Peg found, but no spring...


the SATA backplane with the housing taken off.


SATA rear housing & 4x 80mm fans (which are REALLY LOUD)


Case Front. In the upper left corner is a nifty display.


Rear panel:


I did fire it up. It POST'd just fine, 4GB of memory showed up. Processor is a 2.8GHz Opteron 254.


That's all:


I'll leave the 16 port 9550, but presume the battery for the BBU to be toast....but thar ya go.

[ratdude747]

Re: Ratdude's 3U Server Shenanigans

Bought fresh SATA/SAS cables.

Looking for the backplane. Close and might work with some soldering:

http://www.ebay.com/itm/Set-Rackable...MAAOxyRhBSzsoi

SATA my ass:

http://www.ebay.com/itm/Lot-of-4-Rac...sAAOSwRgJXjiHP

It's ultra 320 SATA!

I might have to see if SGI has one kicking around they could sell me...

[ratdude747]

Re: Ratdude's 3U Server Shenanigans

Brought her home. Pictures.

First, Topcat totally ripped me off, as the case was empty :



Just kidding. I had to remove the board to repair the southbridge heatsink. It's all here. Thanks Topcat! (x100, this thing kicks ass)

Some assclown was here... apparently they thought thermal compound is Peanut Butter:



I also found two different kinds dried up in there... what a mess

Also, that is a wannabe dynatron. Luckily I had two real ones (more fins) to install.

Board and goodies installed:



Those are real dynatrons with a pair of spare Opteron x2 285's installed. Also my zip tie fix for the heatsink. I had a couple spare push pins, but not even the old one with a new spring would grab the board. I've done this to Slot 1 PIII's before with success, so I think it will do fine here.

Installed in rack:



The brackets seemed to be mounted backwards, so I flipped them for a flusher and tighter fit. My poor rack is holding it, but I'm sure it doesn't like it. The again there are 12 10-32 screws with washers holding it in there, so it's probably fine.

---

Drive wise, I won an auction for 12 Seagate 7200.11 250GB drives (the single platter slim kind) for under $100 shipped. I also best offered three more for $11 a pop shipped and one more for $12 shipped. Not as big as I really wanted, but in a 14 drive RAID 10 with 2 hot spares I think it will run great. Thoughts? (And yeah, the 9550's battery is hosed. Trying a spare...)

---

Another thing I will try on this is LAN bonding (similar/same as teaming in Debian), since I have 3 onboard NICs and I don't need gigabit if I ever make it a router/firewall for the WAN connection.

---

Another change that's pointless... well, just read this old post:

https://www.badcaps.net/forum/showpo...5&postcount=75

This board is more or less the same thing BIOS wise (AMI, same platform), so I should be able to make a quick easy swap? We shall see...

---

One other thing I might do with this is make it a local Minecraft server. Just a thought though.

[diif]

Re: Ratdude's 3U Server Shenanigans

It's 7200.14 drives you've bought not 7200.11. Your drives are newer and support SATA 6Gb/s.

[ratdude747]

Re: Ratdude's 3U Server Shenanigans

It's 7200.14 drives you've bought not 7200.11. Your drives are newer and support SATA 6Gb/s.

Ah. Explains a lot (like the AF logo). Still good drives?

[diif]

Re: Ratdude's 3U Server Shenanigans

I haven't needed to worry about hard drives in a while but looking at the specs I'd say so. Faster than the ones you thought you'd bought.

[ratdude747]

Re: Ratdude's 3U Server Shenanigans

I haven't needed to worry about hard drives in a while but looking at the specs I'd say so. Faster than the ones you thought you'd bought.

Gotcha. My controller is only SATA II but I doubt that will bottleneck these.

[Per Hansson]

Re: Ratdude's 3U Server Shenanigans

Drive wise, I won an auction for 12 Seagate 7200.11 250GB drives (the single platter slim kind) for under $100 shipped. I also best offered three more for $11 a pop shipped and one more for $12 shipped. Not as big as I really wanted, but in a 14 drive RAID 10 with 2 hot spares I think it will run great. Thoughts? (And yeah, the 9550's battery is hosed. Trying a spare...)

I was going to recommend you to make sure the firmware is the latest, unless you want to try this fix in a few months
https://www.badcaps.net/forum/showth...264#post106264

But if it's not 7200.11's then no worries

[ratdude747]

Re: Ratdude's 3U Server Shenanigans

For the fans, would something like this make sense?: (only beefier, I have 4 1.53A fans in the system)

https://www.newegg.com/Product/Produ...9157-_-Product

Edit- No luck, I'm out luck, unless I bought one of those and modded it. But if I'm going there, I'd be tempted to design my own PCB and make one that used one of the CPU PWM signals for fan control (via darlington of MOSFET). Thoughts?

And the Dynatrons I could probably swap for PWM fans, as the CPU fan connectors do support PWM.

[ratdude747]

Re: Ratdude's 3U Server Shenanigans

A few hours on KiCAD and...

https://oshpark.com/shared_projects/mWDGPILr





All this does is take one or two PWM signals and amplifies them to 3 or 6 outputs though a TIP122. It's set up as two separate channels, with a jumper to link them if only one PWM signal is used. The idea being that since I have PWM control on the CPU coolers, I can tap into that (or another PWM controller, they do exist on the net but are not high output). See attached schematic.

It uses a molex plug for power (like the old breakout board), standard 4 pin PWM fan headers for the PWM source (that way all I need to do is run a cable to the port, and another to the Fan unless the fan reaches by itself)... I'm sure some cheap fan extenders (if one can buy such) and some splicing/crimping would do the trick.

To keep the TIP122's cool, I designed it to use one of these:

http://www.digikey.com/scripts/DkSea...mSeq=235228169

Hence the layout I chose.

Something I didn't do but though about was adding a switch to short the PWM in to the 5V on the molex to force full blast... but if I need to do that, I'll swap back to the old breakout.

Look like a winner? Edit- Maybe not, it looks like I tried to make it PNP, not NPN. That's what I get for using schematics online. Boards were not ordered. Yay Respin time?

[ratdude747]

Re: Ratdude's 3U Server Shenanigans

If you first don't succeed, call it revision 1.


Here's revision 2, which acutally is NPN as called for in the intel PWM spec:

https://oshpark.com/shared_projects/C1SdRzD4





I also added a 5V full speed force jumper (JP2). It forces channel 2 to full, and also channel 1 if they're linked.

Better?

[Topcat]

Re: Ratdude's 3U Server Shenanigans

^
I take it this is an attempt to quiet this monstrosity down?

[Topcat]

Re: Ratdude's 3U Server Shenanigans

^
I take it this is an attempt to quiet this monstrosity down?

[ratdude747]

Re: Ratdude's 3U Server Shenanigans

Yes and yes.

[ratdude747]

Re: Ratdude's 3U Server Shenanigans

Made a slight change (fixed fan input labels). Ordered boards and components.

[ratdude747]

Re: Ratdude's 3U Server Shenanigans

Also, to save money and trouble, since I have a spare port on each channel, I'll probably run the CPU fans off of this as well.

[ratdude747]

Re: Ratdude's 3U Server Shenanigans

Stuff has come in! Pictures:

First attempt at SATA cables:



The metal clips on the plugs didn't agree, so I axed them, which helped a lot:



Cables routed:



Topcat, you forgot the HDD screws... Turns out Ace hardware had competitive prices on them (in stainless too!) so I bought 200 just to be safe (since pretty much all caddies use these, and I hate torx screws):



While I was near my parents house, I swung by and nabbed some goodies:

• an old 3ware 9500 with a good battery (unlike the two I've tried and reported as "weak" and wouldn't charge)
• another Molex to SATA adapter
• some NEC USB 2.0 cards
• Floppy disks

So I installed the drives, 14 of the 16 good (one of the lot of 12 failed SMART, one of the singles wouldn't spin and is being returned). I also pulled the battery from the 9500 (which I don't care for when I can drop in a 9550 and get a 10mb/s speed boost and also get SMART checking).

Threw in an NEC USB 2.0 card, as this board only has 1.1 oddly (AMD 3100 chipset limitation)

As a test, I moved over the drives from my crappy server, and while it did run, I got some ECC errors which eventually ended in a Kernel Panic. Memtest 86+ (Booted from USB floppy) doesn't report any errors but will randomly reboot. Bad RAM?

Final state tonight:



Progress?

[ratdude747]

Re: Ratdude's 3U Server Shenanigans

UPDATE:

Bad RAM is confirmed. Ran it off 8gb pulled from Main Rig V2.5, rock solid.

Time to get more RAM I guess.

[Topcat]

Re: Ratdude's 3U Server Shenanigans

^
I told you... Sorry about the screws....they're sitting here in a little medicine cup. Makes up for the 2 dead CMOS batteries in the SM blades.