Looking for a NAS

[Dannyx]

Re: Looking for a NAS

Granted, I can serve up more than 400MB/s from "internal" drives -- but, that saturates all four Gbe interfaces.

I assume this is over "cable", since I can't imagine a wireless client going that fast.....if it did, that would be great though

When you say 4 interfaces, are these on the same machine and doing LACP ? (if that's what it was called when you combine more than one to increase throughput). This would require a compatible switch at the other end as well, which I DID look into at one point, but they're hella expensive for my purposes....

[Curious.George]

Re: Looking for a NAS

Isn't USB....slow ? It is, hence why you said this

I can pull 70MB/s off of a "consumer" external USB(3) disk drive.

Granted, I can serve up more than 400MB/s from "internal" drives -- but, that saturates all four Gbe interfaces.

[benchmarking]

What would the best approach for this purpose be ? Is there a "tool" for this sort of stuff, or do I just use regular everyday tasks that I perform with my files as a means to gauge the performance, such as copying files back and forth between a client and the NAS, opening them, overwriting them after editing them and, my favourite, waiting for the thumbnails to generate and reorder by date - this takes A LOT for some reason, even though ironically I CAN open the files just fine in the time it takes for explorer to finally reorder

Don't be so formal. You don't really CARE what the "numbers" are. What you want to do is characterize the user experience -- when you try to access the content IN WHATEVER TYPICAL MANNER, are you happy with the latency and bandwidth that you see? Or, are you grumbling (because the drives need to spin up, etc.)

If you like looking at thumbnail views, then take the time to build ALL of the thumbnails once -- so they are ready and waiting when you go looking into the folder.

[As I said upthread, I use ThumbsPlus for this as it's whole focus is that of making visual selection of images easy. So, it keeps the thumbnails in a database that it maintains.]

In my case, I'm usually pulling entire ISO's or tarballs off of the drives -- 600MB - 50GB. So, I'm more interested in the bandwidth than the latency (it might take me 10 minutes to figure out which drive has the file I want and get it mounted; but, if I can only pull data off it at 20MB/s, I'd be REALLY annoyed!).

If, OTOH, you're looking to click on dozens of photos, one at a time, the "transport cost" may be small and the latency may be the driving annoyance factor.

[Dannyx]

Re: Looking for a NAS

You'd opt to not let windows "sleep" (nor hibernate) if latency was an issue.

It's the WD NAS box in this case that's spinning down - not a Windows PC, though the idea's the same for both. I purposely set the NAS to power down the disks after 15 minutes at the sacrifice of having to wait for them to spin back up, so I am fully aware of the issue, but I figured this drastically increases the overall life of the drives, plus the heat buildup inside that tiny closet.

Then you'd have to ensure you had a comparable wall wart/brick available to swap into its place when/if it shits the bed.

I do - I'm a repair tech, so my house is like a warehouse full of this cr@p, plus a truckload at work, so a wall-wart is as common as it can get

If overall performance isn't a driving factor, then consider a passively cooled "PC" with external USB3 disk(s). If the "PC" craps out, you unplug the USB3 cables and plug the disk into another USB3 (or 2!) port on a bog-standard desktop or laptop and your data is ready and waiting.

Isn't USB....slow ? It is, hence why you said this

If overall performance isn't a driving factor

It's also not the most tidy solution when it comes to cables and junk like that. Not to worry, I have a bunch of SATA>USB adapters, so I don't need to worry about dealing with bare drives (provided of course they're in the proper file system, otherwise you've got a problem !).

But, before you commit to ANY approach, do some benchmarks to get a feel for how things will perform under various usage scenarios.

What would the best approach for this purpose be ? Is there a "tool" for this sort of stuff, or do I just use regular everyday tasks that I perform with my files as a means to gauge the performance, such as copying files back and forth between a client and the NAS, opening them, overwriting them after editing them and, my favourite, waiting for the thumbnails to generate and reorder by date - this takes A LOT for some reason, even though ironically I CAN open the files just fine in the time it takes for explorer to finally reorder

[Curious.George]

Re: Looking for a NAS

I HAVE to wait for the disks to spin up after the box's been in standby. Windows explorer of course has no choice but to hang and wait for this to happen, so this would be one (small?) issue solved by switching over to SSDs

You'd opt to not let windows "sleep" (nor hibernate) if latency was an issue.

Regardless, it will be considerably more responsive than waiting for rust to spin...

No, in this case it's a very small entry-level NAS, so it's powered by a regular wall wart - no worries there in case it quits, unless of course you're thinking about something failing in the INTERNAL DC-DC power converters of the box,

Then you'd have to ensure you had a comparable wall wart/brick available to swap into its place when/if it shits the bed.

Precisely why I started looking into the DIY route, even considering a Windows PC, like I said ! Sure, having a PC in there would be big, noisy, kinda clunky and would take up considerably more room, but I'd be putting to good use a motherboard which is just lying around, it would be a fun build and let's not forget: MUCH cheaper than an off the shelf box, PLUS the disks to support it !

If overall performance isn't a driving factor, then consider a passively cooled "PC" with external USB3 disk(s). If the "PC" craps out, you unplug the USB3 cables and plug the disk into another USB3 (or 2!) port on a bog-standard desktop or laptop and your data is ready and waiting.

[I essentially run this way (though my "PC" runs a custom NetBSD kernel instead of Windows). But, I've been discarding the "external USB enclosures" in favor of a simple USB dock. This lets me store bare drives -- instead of drives that have been packaged in a bunch of plastic (and having to keep track of which wall wart goes with each)]

But, before you commit to ANY approach, do some benchmarks to get a feel for how things will perform under various usage scenarios. You don't want to discover after-the-fact that your solution leaves you feeling cheated...

[Dannyx]

Re: Looking for a NAS

You're assuming that the storage medium is the limiting factor in your latency and/or bandwidth.

I may be wrong in assuming that speed is based solely on sheer disk write/read performance (there's a lot more to it than that, for sure) but one thing that's definitely certain is that I HAVE to wait for the disks to spin up after the box's been in standby. Windows explorer of course has no choice but to hang and wait for this to happen, so this would be one (small?) issue solved by switching over to SSDs.............or not, now that I think of it, there's another possibility: when the disks (or SSDs) power down for standby, the next time they wake up, the NAS has to wait for them to "mount" and it may not necessarily hang because they haven't spun up yet, but because they haven't "mounted" yet, so even SSDs may have the same issue despite their instant "spin-up" time !

lose a power supply (they're usually not just stock "PC" power supplies)

No, in this case it's a very small entry-level NAS, so it's powered by a regular wall wart - no worries there in case it quits, unless of course you're thinking about something failing in the INTERNAL DC-DC power converters of the box, which could very well happen...or indeed this:

encounter a software problem (bug or firmware corruption)

THIS would be an issue indeed !

Part of my reason for moving to bog-standard hardware (and FOSS software) is to ensure there's nothing preventing me from accessing/recovering data regardless of the failure.

Precisely why I started looking into the DIY route, even considering a Windows PC, like I said ! Sure, having a PC in there would be big, noisy, kinda clunky and would take up considerably more room, but I'd be putting to good use a motherboard which is just lying around, it would be a fun build and let's not forget: MUCH cheaper than an off the shelf box, PLUS the disks to support it !

[Curious.George]

Re: Looking for a NAS

Another idea was to have BOTH a SSD box for quick access to "everyday" files and a HDD box for a more permanent storage of important data.

You're assuming that the storage medium is the limiting factor in your latency and/or bandwidth. IME, you'll find that MEASURING that performance will give you numbers that belie that assumption. Don't pay for performance you won't get!

This might be a non-issue however, since I think it's highly unlikely for the NAS box ITSELF to fail (which would leave me with drives stuck in an unusable filesystem, hence the idea) - I imagine the disks inside it are more likely to give up before the box starts hiccuping in any way.

You could bugger a network connector, lose a power supply (they're usually not just stock "PC" power supplies) or -- worse -- encounter a software problem (bug or firmware corruption) that effectively makes your data inaccessible.

[I had a NAS years ago that stored configuration information on the spindle so the network connection went down when the disk failed to spin up!]

Part of my reason for moving to bog-standard hardware (and FOSS software) is to ensure there's nothing preventing me from accessing/recovering data regardless of the failure.

[Topcat]

Re: Looking for a NAS

You've also mentioned more than once you don't like patching your OS.

It depends on the patch and what the patch is 'fixing'....but I don't dogpile on every patch M$ puts out these days.....they've proven to be more & more detrimental these days, especially with w10. If it's something severe, I'll still wait until it's broken everyone else's system, then they re-release the patch....I don't knee-jerk patches.....patience is a virtue! My NAS, I don't patch it much (S2K12 R2), it has zero connection to the outside world...I don't get overly excited about updating it.

SSDs for the OS (RAID), spinning stuff for the data.

Yup.

[Dannyx]

Re: Looking for a NAS

Another idea was to have BOTH a SSD box for quick access to "everyday" files and a HDD box for a more permanent storage of important data. I imagine there exist boxes with 4+ drives that have this setup in mind and provide automation ("syncing"), though I could probably rig up something myself from scratch, say: use my existing WD box for two SSD(s) and a PC for 1/2 HARD disks. I imagine it would be better to have a Windows machine hosting the "final" version of the data, since if the machine fails, I have a usable filesystem on the disk ready to go on another PC with absolutely no complicated recovery scheme required. This might be a non-issue however, since I think it's highly unlikely for the NAS box ITSELF to fail (which would leave me with drives stuck in an unusable filesystem, hence the idea) - I imagine the disks inside it are more likely to give up before the box starts hiccuping in any way.

[diif]

Re: Looking for a NAS

I just have a hard time trusting critical data to a SSD for some reason. I keep good backups of mine....but unless you dump it with every change (which for me would be daily), there's far greater risk for data loss. I usually do a full dump once a month and depend on the RAID (probably more than I should) in the mean time.

You've also mentioned more than once you don't like patching your OS. Yes we used shadow copy in the day, incremental backups every night and a full backup every week. Rotated and taken off site. As long as the user clicked save, nothing was ever lost.
SSDs for the OS (RAID), spinning stuff for the data.

Same...would not use them in a NAS myself, hence why I asked if it's common practice in the data center industry...probably not.

There is plenty of IT in between a home user wanting to share 500GB on a network and data centres. Depending on the speed required there are large flash arrays but the really quick stuff is run all in memory.

[Dannyx]

Re: Looking for a NAS

I just have a hard time trusting critical data to a SSD for some reason.

Same...would not use them in a NAS myself, hence why I asked if it's common practice in the data center industry...probably not.

[Topcat]

Re: Looking for a NAS

There are enterprise SSDs, but even a standard desktop SSD from a reputable manufacturer would suit most situations.
NAS drives are more resilient to vibration and heat when they are densely packed. Not something SSDs worry about.

I just have a hard time trusting critical data to a SSD for some reason. I keep good backups of mine....but unless you dump it with every change (which for me would be daily), there's far greater risk for data loss. I usually do a full dump once a month and depend on the RAID (probably more than I should) in the mean time.

[diif]

Re: Looking for a NAS

Do SSDs play well with NASs or are there some SSDs specially designed for this ? Sorry if I already asked this - I forgot if I did

There are enterprise SSDs, but even a standard desktop SSD from a reputable manufacturer would suit most situations.
NAS drives are more resilient to vibration and heat when they are densely packed. Not something SSDs worry about.

[Dannyx]

Re: Looking for a NAS

Reliability.

If you want speed, attach an SSD to what you already have and share it.

Do SSDs play well with NASs or are there some SSDs specially designed for this ? Sorry if I already asked this - I forgot if I did

[diif]

Re: Looking for a NAS

What do we look out for in NAS-grade HDDs ?

Reliability.

If you want speed, attach an SSD to what you already have and share it.

[Dannyx]

Re: Looking for a NAS

Red is 5400 RPM

Well damn, you're right - they are....I would've never thought something so "high-end" would be 5400. I thought there's no way it'd be anything lower than 7200....wasn't expecting 10k RPM, but still....5400 ? Isn't that like a baby's toy ? Ok, there's probably a lot more to it than that...I'm not an idiot, I realise they've got an "impressive" (?) cache of 256mb which is a lot (?) compared to regular "desktop" drives.

REs (not to be confused ! ) are 7200 RPM but 64mb cache....soooooo....yeaaaah.....which ones do we choose ? What do we look out for in NAS-grade HDDs ?

EDIT: scratch the first part - what I saw is the WD Red PRO which are indeed 7200 RPM...

[Uranium-235]

Re: Looking for a NAS

Which one is that ?

Red is 5400 RPM. RE4's (technically discontinued) are 7200 RPM. RE's are technically server grade

price is the only issue but they've gone down since being discontinued.

[Curious.George]

Re: Looking for a NAS

I don't transcode. But I did want server grade stuff for a while years ago but it's grown pointless.

However a Synology NAS DOES transcode, and i'm looking forward to that

Why not just "recode" your content for the devices you intend to serve?

E.g., my audio collection exists in three different "encodings" for the three different types of uses I make of it (MP3 for PMPs, FLAC for my audio server and the original "rips" from the CDs).

I'm always leary of relying on functionality embodied in some CLOSED appliance. Codecs/containers evolve, over time, yet there's no guarantee that the appliance's software/firmware will even attempt to keep up with those changes.

[Dannyx]

Re: Looking for a NAS

No, raid editions outperform red

Which one is that ?

[Uranium-235]

Re: Looking for a NAS

I'm definitely going with WD too, though I believe you meant to say RED Will probably kick it up a notch with 4 drives, or.....NAS SSDs anyone ?

No, raid editions outperform red

[Dannyx]

Re: Looking for a NAS

I might end up going with an off the shelf NAS from Synology too. Sure, it is expensive and might not give you the same satisfaction as making one yourself, but it also comes down to two simple facts: the space and the overall performance. There's not a lot of room available where I store my network junk, so a PC would be larger and noisier, giving a NAS a box the advantage. An OTS NAS is surely better engineered and thought out that anything I, as a standard home user with modest skills and budget, could put together. I also managed to smash my SGS3 a couple of days ago (still working, but the glass's cracked), so that will set me back quite a pretty penny to get a replacement....just as well really, because a new phone was the last thing on my list and I was going to stick to the SGS3 till death do us part....now it finally has, so I'm finally, against my will, getting a new phone

Another idea was to decommission that lone torrent machine and replace it with one that runs BOTH the NAS and the P2P stuff. Trouble is it might be very slow and inefficient, because even with different disks ("spindles") you've now got the same CPU, same storage controller and same network interface serving TWO "hungry" services. A bog-standard "office" MB like that is just not cut out for the job, even with the Xeon currently on it...probably comes down to the chipset and network equipment AFTER it. A home router is also bottlenecking pretty hard I imagine...