Server Upgrade?

[eccerr0r]

Re: Server Upgrade?

I think the VGA on my new-to-me SM board is dead... does not seem to init. At least 3 of the 4 gbit lan ports, sata3, 1 of 4 sata2, and PCIE x8 looks ok, have not tested the x4 or a lot of other stuff...

Still trying to get a decent Gentoo install for it... taking a while since it's an atom cpu.

Incidentally, after it being a lot slower than i expected it to be despite being a silvermont quad core, I'm really not sure what I'll do with the 16GB RAM in the machine. Too bad it's 4x4, probably why the board was cheap, if it had 8's I'd stick it in my other ddr3 machine...

(I knew my bonnell/diamondville atom was really slow, this silvermont isn't way too much faster. almost double speed per core but still like half the speed of my core2 quads that have half the RAM... was thinking about using the atom as my server just so I could use 16gb ddr3 ECC RAM instead of 8gb ddr2.)

[ratdude747]

Re: Server Upgrade?

Fresh GLAN cards came in. This time the 1x PCIe 2.0 variety.

That said, I think the MOBO is dying... as afterwards my PCIe x16 and PCIe x8 slots are dead. Not even 1x cards are seen. Running the 3ware card in a 1x slot to get by. Might explain the dead onboard GLAN... Shame, SM boards are supposed to be bombproof???

[ratdude747]

Re: Server Upgrade?

Got it running on Arch. Copied the laptop install and modded it from there (running XFCE and lightdm, tried LXDE but it was too limited to be useful). One of 2 GLAN's is working... the other doesn't even show up. The jumpers are set right, so I'm thinking it's fried. Also confirmed that the dual GLAN card is a paperweight ... will source another.

Oh, and I posted this from it. One of the perks of running a GUI

[ratdude747]

Re: Server Upgrade?

Good score on that keyboard. They have any more?

Nope. I saw one form another seller for $200...

Most of the keyboards I saw were full consoles (pull out dumb laptop-like thing with an LCD) for $70+ (many were practically free before shipping).

I'd fresh install, IMO. Better to have a known clean server install than try to carve a desktop *nix install into a server one. Way too much dependencies and crap left behind on a stripped dirty install.

Anything other than Arch I'd tend to agree. Arch can be a bit of a pain to clean install (at least doing it the way I was taught)... copying is way easier if the architecture matches. I've used the same install on 6 laptops across 5 SSDs since 2014 or so (and other install on a handful of ones before I consolidated due to Nvidia drivers going to hell). All I do is set the new partitions, mount partitions and copy from the "donor" via a live arch stick, redo the fstab, chroot, change the name, reinstall the kernel, and install/generate GRUB (EFI or MBR). Other than some uneeded programs, most of what my laptops run is similar to what the server would run. It's all intel graphics on AMD64.

[Topcat]

Re: Server Upgrade?

Good score on that keyboard. They have any more?

[TechGeek]

Re: Server Upgrade?

More stuff came in last night. HDDs are in... easy enough. I have two hot spares (all 16 bays filled) and two offline spares. Should last me a while... or at least until NVME raid cards (and NVME SSDs) get cheap enough that I don't have to fork over $500 for such a setup (which is what the current going rate on eBay seems to be). Until then I don't see a point in spending money on a new drive setup... save that for a much larger change (1u/2u chassis swap, etc.).

Also, the keyboard came in... yep, actually a new in box unit:



Installed:





The only downside is that it's big enough to get in my way when working in the closet. Have to back up a bit. But, sure beats the cheapo USB mini-keyboard I was running.

Also, I'm probably going to run arch on this after all. It's what I'm used to on the laptop side, and honestly I haven't had too bad of luck the last 8 or so years I've been using such. Not to mention it wouldn't be that hard to take my laptop install and port it over and then mod it into a server setup (remove gnome, keep cinnamon, reset network interfaces, etc.)... and a lot of the custom files I used on the old debian setup likely will port over, or at least that's my hope. Or heck I might do a fresh install... I dunno.

I'd fresh install, IMO. Better to have a known clean server install than try to carve a desktop *nix install into a server one. Way too much dependencies and crap left behind on a stripped dirty install.

[ratdude747]

Re: Server Upgrade?

More stuff came in last night. HDDs are in... easy enough. I have two hot spares (all 16 bays filled) and two offline spares. Should last me a while... or at least until NVME raid cards (and NVME SSDs) get cheap enough that I don't have to fork over $500 for such a setup (which is what the current going rate on eBay seems to be). Until then I don't see a point in spending money on a new drive setup... save that for a much larger change (1u/2u chassis swap, etc.).

Also, the keyboard came in... yep, actually a new in box unit:



Installed:





The only downside is that it's big enough to get in my way when working in the closet. Have to back up a bit. But, sure beats the cheapo USB mini-keyboard I was running.

Also, I'm probably going to run arch on this after all. It's what I'm used to on the laptop side, and honestly I haven't had too bad of luck the last 8 or so years I've been using such. Not to mention it wouldn't be that hard to take my laptop install and port it over and then mod it into a server setup (remove gnome, keep cinnamon, reset network interfaces, etc.)... and a lot of the custom files I used on the old debian setup likely will port over, or at least that's my hope. Or heck I might do a fresh install... I dunno.

[ratdude747]

Re: Server Upgrade?

I've done some stuff but have been too swamped/zombified by work and other things to post about it.

The good news is that three cards later I finally have my RAID array back (1st card only had 12 ports, 2nd was completely faulty with PCB damage). But, two drives died... thankfully in a way that the one hotspare I had (forgot I had a 15th drive installed) could be rebuilt for one of them... but one mirror is degraded. I've ordered a lot of 5 drives for $30 shipped to get back to 16 drives (14+2 hot spare) and still have two drives left over for cold spares. Also, no errors or issues out of the 80GB HDD... maybe it was a bad ATA to SATA adapter after all? Not that it matters since I'm going to SSD it anyway...

The bad news is there's some other issues:

-Turns out that the PWM driver I designed in the prior thread does work (which is why it was installed some time ago)... it just never seemed to do much because neither Mobo I used did much with the PWM. In fact, it works so well that I put the original 6000RPM fans back in... but, it works too well in that it keeps ramping the fans down below stalls speed; if the tach signals are connected the board sees this and bumps it up to full blast and then winds it back. Don't connect the tach's, the fans stall. Heck, I'm hearing it do this a floor below in my bleeping garage. They only give 2 speed options in the BIOS- Normal (50%) and Full Speed; supposedly one can use the BMC/IPMI controller to customize that but all testing and info points to this board not having such in a way I can access (as there's no settings for such other than serial console redirection and it's only mentioned in the manual explaining how the fan control system works). Dunno what I'll do here. Maybe install 7/10 to a junk drive long enough to get on Watchdog III and see if there's a setting in there? The IPMItool program debian supports didn't work (and no IPMI device is showing as existing; that may be a hardware issue though).

-I have no GLAN. The marvell card doesn't show up at all (not even on lspci, which makes me think it's dead), and only one of the onboard GLANs shows as an interface (and it's running in 10/100 mode). Per some research, apparently Debian doesn't support the right driver for the intel e1000e series ethernet used, and while somebody has a Github with a DKMS .deb driver, it won't install (I upgraded to Debian stable, I think it requires Testing???). Supposedly Ubuntu and other less backwards distro's do support the driver. Maybe Debian isn't the way to go after all?

-Since I have a quasi-decent GPU built into my CPU (and I have Displayport and HDMI, not just VGA), I'm thinking if I swap distro's, a light GUI might be handy for certain bits of maintenance (direct copying stuff over USB3, etc). Until I wiped it tonight, I was actually running my arch install on the old laptop SSD for debugging. I also found a rack sized addesso keyboard/trackpad combo in my junk pile (upgrade for the USB mini keyboard I keep on the rack shelf)... but it has switch issues and isn't repairable. Undeterred, I did find a supposedly used (in box/wrap???) Cherry LCDK1031-BBKU rackmount keyboard and trackpad assembly for $40 (best offered down slightly) that I'll be installing under the monitor shelf.

Pics:

I did a bit of case modding to bring it up a decade... Drilled out a slim floppy/DVD filler into a set of front USB3 ports:



I ordered header cards for this. Sure, I could have saved a couple bux and ordered just headers, but the brackets can be used as templates for drilling/filing the holes. I flattened the bracket (and bend a new tab for spacing reasons), then match drilled and riveted it to the filler to allow for drilling and filing the holes for the ports themselves:



(Yeah my bench is a mess... like I was saying about being busy and zombified)

Once done, I then drilled out the rivets and...



(yeah the picture quality is potato )

Installed:





Pics of the trashed 3ware card I'm sending back for a refund:





I guess by "tested", they meant "it POSTed". As only 8 of the 16 ports were working and I have a bad feeling it may have fried the two bad drives (as they didn't come up as failed with a bad config/head clicking until after trying this card). Yes I did pull the drive boards and clean the contacts... no bueno.

---

All of this considered, I need to figure out what distro to run here. I'd go with Arch if I didn't think I'd forget about updating it often enough (as arch is what I use on my laptops, and when I ran an old install as a test, I do remember having gigabit enabled). Debian is stable... but this LAN bullpuckey is a bit of deal breaker (unless I fork over more $$ on a different LAN card with better support). The other distro that comes to mind would be CentOS... but I haven't run an RPM distro since my days of running Mandriva in College (before I swapped to Arch and never looked back for non-server use).

[ratdude747]

Re: Server Upgrade?

RD, I'd look into a LSI 9260, they're fabulous controllers, support SAS (SSD SAS as well), and can be had for peanuts now, including a BBU with a good battery.

No good for the provision of keeping the existing chassis. The issue here is that the existing backplane is SATA, requiring 16 discreet SATA connections. SAS cards don't work since they usually only give 4 or 8 connections (1 or 2 mini-SAS), with the expectation that one will be using SAS expanders (integrated into such SAS backplanes) for higher count arrays. Since pure SATA RAID was a 3ware specialty, that limits me to such cards/controllers... hence the move from a 9550 PCI-X to a 9650 PCIe.

Otherwise, I'd have gone for a newer LSI (I have several 8708EM2's in service) and started fresh from the backup drive's most recent backup image.

Edit- I see discreet SAS expanders are a thing after all... but they're an additional PCIe card to be used with an existing SAS controller (or another card). I already have another 9650 on order, so it's moot for now, but maybe I wasn't as out of options as I thought?

[Th3_uN1Qu3]

Re: Server Upgrade?

Here we go:

https://www.ebay.com/itm/234099727949

A bit spendier than what I would have wanted, but I think this will fit the bill. Yeah, it has a 65W i7 in it, but if I want to cut that to 35W, it'll accept a Xeon E3-1265L v4. 25W, there's the Xeon E3-1240L v3. Heck, if I only want 2 cores, they made a 13W E3-1220L v3. Will buy if eBay's site stops glitching out.

Nice. Solid choice. Just as solid as when I convinced Chris to buy a Ryzen consumer system instead of a dual CPU Sandy Bridge server. Let the old geezers get some rest. Soon they will be material for CPU Museum.

[Topcat]

Re: Server Upgrade?

Tons of older drives can be had for pennies, I have 4 boxes (around 400) of good drives ATA and SATA, I'm sure Topcat has a room full.

I sold about that many to a local guy, all 500gb and under for $100. 750GB~4TB are fewer, but they're breeding nowadays. Old Rust drives are pretty darn worthless. Once upon a time I'd have killed to have a stash of HDD's like that!!

While I preach SSD's endlessly as an upgrade, I preach backing up data even hard; we all know what happens when a SSD fails.....but they're not as failure-prone as they used to be....atleast if you buy a quality drive.

RD, I'd look into a LSI 9260, they're fabulous controllers, support SAS (SSD SAS as well), and can be had for peanuts now, including a BBU with a good battery.

[ratdude747]

Re: Server Upgrade?

Tons of older drives can be had for pennies, I have 4 boxes (around 400) of good drives ATA and SATA, I'm sure Topcat has a room full.

And I have several on the shelf too. Again, this wasn't a "I couldn't fix the immediate issue" problem as much as it was a "should I really be pushing 20 year old hardware that's in many respects obsolete?"

---

Mobo and RAID card came in. Haven't unboxed the mobo yet but the raid card is a fail. They sent me a 12 port card, but a 16 port card was listed and pictured. A 12 port card won't do when I have 14 drives

I do have a 4 port 9650se though... which I just pulled the BBU off of. I know some cards can be bridged... not sure if such could be a fallback here. Will look it up (and contact the eBay seller in the mean time).

Edit- the spec sheet from LSI says they can be linked... plan B? Edit2: However one does it, it's not working between a 4 and a 12. Neither card recognized eachother. And the 9550 card requires a 3.3V PCI, which I don't have (all are 5V, so no cheating that way). Guess I get to hunt down another card Edit3: Found another one. Closer to me and $5 less. Go figure

[brethin]

Re: Server Upgrade?

All the drives I have are fully tested and pass manufacturers diagnostic, no smart warnings.

[brethin]

Re: Server Upgrade?

Tons of older drives can be had for pennies, I have 4 boxes (around 400) of good drives ATA and SATA, I'm sure Topcat has a room full.

[ratdude747]

Re: Server Upgrade?

I get confused on what your objective is other to buy more stuff, what function was the old server not providing that you felt you needed to upgrade it?

Looking to future proof a bit and improve reliability without throwing the baby out with the bathwater. Yeah a replacement boot hdd would have fixed it... But the fact that I'm limited to older drives is evidence of the bigger issue.

[brethin]

Re: Server Upgrade?

I get confused on what your objective is other to buy more stuff, what function was the old server not providing that you felt you needed to upgrade it?

[ratdude747]

Re: Server Upgrade?

Regarding the "old" server: Did some testing today, and made some findings:

-The place I was thinking of mounting the SSD doesn't work. I'll probably just use the existing boot drive spot. Since I'm missing the hotswap board for that bay (and I can't be bothered to spend $100 on a replacement) it's not biggie. I even have an adapter that will allow the use with such a hot swap board if I ever find another for a price that's not highway robbery (this is the first time I've actually seen one in the wild, although I haven't been looking too hard).

-I do need 2U cooler clearance due to where the rear hard drives live. Hoping the stock intel cooler will work... if not, then I'll hunt a dynatron or the like.

-The existing PATA to SATA adapter doesn't like the Intel SSD (which is a 545s, not a 5400 ). Had a range of results between no POST and GRUB whining about HDD access out of bounds. And BIOS couldn't see the drive in any case it did POST.

-While doing some final testing, discovered that the ATA errors are in fact a failing boot drive. . Between that and the SSD incompatibilty, I've shut down the server and likely won't fire it up unless I must. Thankfully, the 4TB image drive does have a backup image of the boot drive that I may need to use.

The last point brings up a question, and that's on migrating the system (Debian) to the drive. I've migrated/cloned arch installs bajillion times, but the few times I've done such on Debian, have only ever used cloning software or done a fresh install and try to copy over settings each time. Not sure what I want to do here... Not thinking of using arch since arch isn't "stable" (although in laptop use I've hardly had issues). Also debating on whether I should stick with MBR or move to GPT (since the new board is UEFI compatible- moving now ensures future upgrade compatibility). I've done that migration on arch before (when I first set up the Lenovo T460s I'm typing this on), but never on Debian... Maybe do a fresh install, reload from the existing package list, and copy over my /etc (other than fstab, obviously)?

[ratdude747]

Re: Server Upgrade?

Here's a comparison of my CPU options:

https://www.cpubenchmark.net/compare...on-E3-1220L-v3



Well then. The first one is my existing setup (only as a twin-CPU setup). The second is what's already in the board I ordered. And the last 3 are the best performing chips in LGA1150 for the various wattages (Broadwell only went down to 35W, the 25W and 13W options are Haswell). If these benchmarks are true, to maintain my current performance anything quad core is a winner... but if I'm OK with losing performance (which probably isn't an issue unless I ever go back to hosting game(s) on it), the 13W special is viable.

That said, E3-1265 V4's are unobtainium it seems (no "cheap" ebay listings), so that can be scratched off (and the other Broadwell 47W chips are the same way). And E3-1240L V3's can be found in China for $52 ($69 in the USA). E3-1220L V3 is $35 in the USA... so in terms of $$$ a CPU swap doesn't completely pay-for-itself (if the benchmark annual cost to run is accurate for my area- I really don't know). So, for now, I'll probably leave the 65W i7 as-is unless I run into heat issues (or the financial incentives change).

[ratdude747]

Re: Server Upgrade?

Nice! Your love of SM brings a tear to my eye! I've trained you well!

I did look at a lot of other things. Just happened to be another SM that struck my fancy.

If I remember correctly from your thread many years ago, that board uses an nVidia chipset. If so... I wouldn't be surprised with this result at all, regardless that it's a SM board. Seems like anything with an nVidia chipset is bound to give trouble at some point. For this reason, I would never use a board with an nVidia chipset anymore as a main system or anything that needs good reliability. Beater system? - Eh... maybe.

Wrong dual 940. This one is all AMD chipsets (SM H8DAE). Which means no USB 2.0, PCIe, or native SATA. Or an AGP bridge that is supported by anything that isn't Windows XP x32 (or older) .

The system you're thinking of I still have but not currently in use (Main rig V3.8). Only retired because it was stuck on Windows 7 (and due to being SLI "legacy" Nvidia GPUs, wouldn't run up-to-date linux at full power).

https://cdn.badcaps-static.com/image_cache/0790d61b6e661ae21bcb29502c3ef613.gif[/IMG]
Probably makes more sense in the long run at this point than going with something older.
The board looks quite well built (well, it's SM and with older Intel CPU + chipset) so it should be quite reliable.

"Older" but newer than any non-laptop I have in service (Haswell, Can accept broadwell chips too).

[momaka]

Re: Server Upgrade?

The main issue I'm having is random ATA errors locking the system up at the most inopportune times. Probably the PATA to SATA adapter

If I remember correctly from your thread many years ago, that board uses an nVidia chipset. If so... I wouldn't be surprised with this result at all, regardless that it's a SM board. Seems like anything with an nVidia chipset is bound to give trouble at some point. For this reason, I would never use a board with an nVidia chipset anymore as a main system or anything that needs good reliability. Beater system? - Eh... maybe.

Bought the above board/CPU/RAM/cooler. Also nabbed a dual port Gigabit card for 10 shipped.

Probably makes more sense in the long run at this point than going with something older.
The board looks quite well built (well, it's SM and with older Intel CPU + chipset) so it should be quite reliable.