Ratdude's 3U Server Shenanigans

[ratdude747]

Re: Ratdude's 3U Server Shenanigans

Look what I scored:

http://www.ebay.com/itm/Silicom-PXG4...r/172667795492

Offered $10 a pop for 3 cards, automatically accepted (makes up for all the annoying auto-declines we come across?).

Now what do do with cards #2 and 3... I was going to upgrade my main rig, but the motherboard layout combined with my USB3 and GPU cards doesn't allow it. Hmm...

[Topcat]

Re: Ratdude's 3U Server Shenanigans

Not bad! I'm not, nor have ever been a fan of Tyan.

[ratdude747]

Re: Ratdude's 3U Server Shenanigans

TYAN SUCKS!!!!

The damn PWM apparently was an afterthought (only on later revision boards, which is what I have) and thanks the the bios not taking an upgrade, I doubt it ever will work.

So, I've decided to scrap that board and instead get one of these:

http://www.ebay.com/itm/272396459503

Yeah, $22 shipped. Supermicro or die!

It has PWM ports out the ass, but due to the fan system I'll still need to use my controller board (unless I wanted to rewire the entire fan harness to the case, ugh).

So, that's another POS tyan for ya. I'll run it until this other board comes in.

[ratdude747]

Re: Ratdude's 3U Server Shenanigans

Got minecraft working on the server... Port forwarded. I won't post my IP but if anybody who I "know" on the forum wants it, PM or email me.

Only bummer is that it isn't liking my existing minecraft worlds I copied over (swapped level name in server.properties, crashes when I start the server). Whoops?

Still, yay minecraft... the server will probably mainly be for showing off my handiwork rather than actual SMP/PVP play.

[ratdude747]

Re: Ratdude's 3U Server Shenanigans

It helps if one remembers the solder mask...

[ratdude747]

Re: Ratdude's 3U Server Shenanigans

Oh Sh!t!!!

I was about to order those from OSH park, and then realized I never test fitted my components into the faulty boards that came in today. Doh!

Turns out everything fits but I had the molex pinout backwards... Respun the boards:

https://oshpark.com/shared_projects/hfD47kWG






And then got sticker shock and decided to try out allpcb.com. They're direct out of China/Hong Kong, and while the board quality is less (not gold like the OSH park ones, these are lead solder plated), I get 5 (or according to some, more since they have a habit of tossing in spares) for less than the cost of 3 and shorter lead time (since I went with DHL). $5 (boards) + $24 (DHL) + $1.25 (Paypal fee, if I wasn't chickensh!t with my CC number that fee would have gone away) is cheaper than the $36 OSH park wanted. Don't get me wrong, OSH part is a great company, but funds are somewhat tight and I'm pinching pennies where I can.

[ratdude747]

Re: Ratdude's 3U Server Shenanigans

Look what I found:

http://www.ebay.com/itm/Silicom-PXG4...MAAOSwB-1Ypiht

Deal or trouble?

Edit- Nah. Reading online that they have lame BSD support, which makes me less than confident about long term linux support. I'm seeing dual port intel cards around $15, so if I ever need one, I can get one. I don't need more than 3gig, so I'll pass for now.

[ratdude747]

Re: Ratdude's 3U Server Shenanigans

Had a genius moment over lunch... and another upgrade:

https://oshpark.com/shared_projects/eVCOeNDj





Changes:

• Added 4 pin header J2 for accessing the Tachometer signals from the other 4 fans.
• Converted all power connections to fill zones to maximise current carrying capacity

This will likely be the last update to the board design before the next order.

Look good?

---

RAM came in today... works great. Yay. Now to nip this fan controller mess in the bud...

[ratdude747]

Re: Ratdude's 3U Server Shenanigans

You need one more...you're one gigabit NIC away from a tie.

Twin port card needed.

[Topcat]

Re: Ratdude's 3U Server Shenanigans

You need one more...you're one gigabit NIC away from a tie.

[ratdude747]

Re: Ratdude's 3U Server Shenanigans

Looks good in a picture....the tell-tale will be if it works.

We shall see.

---------------------



Found my old trusty (and never actually used for anything) Broadcom NetXtreme BCM5703X Gigabit PCI-X 133MHz card in the pile (read: sitting in a dormant Netfinity 7000) and decided to have some fun.

That said, at least with this, the bonding setup is actually different than above... seems all my troubleshooting for 2 ports quit working, as the old file layout actually works:

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

source /etc/network/interfaces.d/*

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
#auto eth0
#allow-hotplug eth0

#iface eth0 inet dhcp


auto bond0

iface bond0 inet static
    address 192.168.1.3
    netmask 255.255.255.0
    network 192.168.1.0
    broadcast 192.168.1.255
    gateway 192.168.1.1
    bond-mode balance-rr
    bond-miimon 100
    bond-slaves eth1 eth2 eth3
    #dns-* options are implemented by the resolvconf package, if installed
    dns-nameservers 192.168.1.1


auto eth1
iface eth1 inet manual

auto eth2
iface eth2 inet manual

auto eth3
iface eth3 inet manual

What was the real problem? Apparently ifup/ifsdown and the networking service do not agree and one shouldn't use use the former in debian. Either that, or the latter does the former when starting/stopping. The "new" file format was giving a bunch of "option primary not supported in balance-rr" errors when the service started, which hinted me towards reverting to the old format and skipping ifup (which did the trick).

Something I wasn't sure about but seems to be doing OK is the usage of two 133MHz cards in both 133MHz slots... as the manual wasn't super clear on on if that would work... they "recommend using slot 1). lspci -vv is reporting 133mhz on all 3 gigabit NICs and also the 3ware 9550 too. So far, so good...

Yay 1/8th of my switch is now used by my server... maybe I should have gone for a 48 port switch?

[Topcat]

Re: Ratdude's 3U Server Shenanigans

Looks good in a picture....the tell-tale will be if it works.

[ratdude747]

Re: Ratdude's 3U Server Shenanigans

... and here we go:

https://oshpark.com/shared_projects/Zx7VlDjY





No more tommy gun via stitching... instead we have hobo patches (fill zones)...

Look Good?

[ratdude747]

Re: Ratdude's 3U Server Shenanigans

Did a crap ton of math to find out that one driver per PWM is the max, and 2.4K is ideal (due to the relatively high 25kHz waveform used).

So, I'll have to put in a 5A rated slide switch instead, as one of the Power MOSFETS can handle 10A in theory.

Fun fun fun...

[ratdude747]

Re: Ratdude's 3U Server Shenanigans

As for why/how I ended up barking up the wrong tree design-wise, I'm blaming the cesspool of bad information online and me missing a few critical design points.

A careful read of the Intel's spec for PWM (see attached file), especially page 9, cleared up a lot of the nonsense. Yup RTFM yet again.

Once I saw the sentence by itself saying "pull up to" in the Intel tech sheet, I saw what was really going on. Hence why I saw a lot of schematics with TIP120's and resistor bridges...

...as for the move to MOSFET, that's because a tip122 only has a minimal gain of 1000. The intel spec limits the PWM input to a fan to 5mA... which for a spec'd load of 10A (a rough number I've used in the design), on both channels linked to one input we'd have a 10mA load. Since we have a pull-up circuit, that 10mA is over 7.3V (4.7V zener on the other side), which would be a 730 ohm ideal resistance. Since the input has to sink that, assuming a perfect 0V CE drop (worst case), that would be 16.4mA! Over 3x the current limit! However, the schematic I found online, when the output sinks, only has at most 4.44mA, which is close but not above the limit (what we want when designing a transistor/MOSFET driver).

Look good?


EDIT- FAIL. I'm going to make some changes.... As the driver will draw too much current to run another PWM signal off of it, I'm going to axe the PWM out ports and the jumpers.... and also move the tach ports and respace so PWM fan connectors will clear. Also seeing if I can half the resistance on the pull up resistors. Ugh. Here comes Rev 5!

[ratdude747]

Re: Ratdude's 3U Server Shenanigans

https://oshpark.com/shared_projects/zH7ELRHW





Changes:

• Conversion to a pulled up zener-based MOSFET Driver
• Removal of Force switch, as the pull up achieves the normal PWM fan function of no input= full blast
• Added a Tachometer passthrough to J8 and J11 via JP2 and JP3, as apparently some systems need the feedback to set base fan speed
• Took a tommy gun to it; due thin spots in the ground plane I decided to via stitch critical areas to increase current carrying capacity

[ratdude747]

Re: Ratdude's 3U Server Shenanigans

Did more looking... the Darlingtons use too much base current.

Looking at mosfets instead... found this:

https://electronics.stackexchange.co...ns-with-mosfet

Apparently when reading the intel spec I had things backwards? If it is indeed open collector, then that's the right circuit. I'll make a mosfet version based on that circuit.

Edit- Yup, confirmed. Mosfets and pullups it is.

[ratdude747]

Re: Ratdude's 3U Server Shenanigans

Another update:

https://oshpark.com/shared_projects/MdnDZqET





Changes:

• Added Screw holes in a 1.5" x 2.5" pattern (6-32 HDD/case screws and standoffs) as my old idea of using foam tape is probably doomed to fail
• Addeda bottom silkscreen layer as it's no longer getting taped (if you have a better message for the bottom, I'm open to funnier suggestions)
• Rechecked a print against real heatsinks to verify clearances (this was an issue with versions not posted)

Look good? As much as I hate it, I won't order until I have one of the V1 PCBs hacked together to verify the schematic.

[ratdude747]

Re: Ratdude's 3U Server Shenanigans

Ok, Here's Rev 3:

https://oshpark.com/shared_projects/V9E8DxQL





Changes:

• TO220 footprint fixed (Damn you KiCAD... and Damn me since I've already been burned on that before )
• Copper fill is ground, not 12V (no more warning needed)
• Swapped JP2 for a DPDT switch, like used on my Supermicro breakout board

I'll sleep on it before I order more... Did I mention I'm a dumbass?

[ratdude747]

Re: Ratdude's 3U Server Shenanigans

If you first don't succeed, call it revision 1.


Here's revision 2, which acutally is NPN as called for in the intel PWM spec:

https://oshpark.com/shared_projects/C1SdRzD4





I also added a 5V full speed force jumper (JP2). It forces channel 2 to full, and also channel 1 if they're linked.

Better?

Bummer... Today I remembered something about KICAD: the TO220 pinout is wrong and pins 2 and 3 are backwards!

For testing at least, I'll probably be able to cross-leg the transistors, I hope.

In the mean time, I'll redo the board design and re-order them.

Doh!