Looking for a NAS

[Topcat]

Re: Looking for a NAS

...Power consumption...

I was actually thinking of reworking my NAS for sake of power consumption....but I just can't bring myself to do it....

[Dannyx]

Re: Looking for a NAS

What do you think of XigmaNAS ? I was thinking of using that with the Gigabyte board. Power consumption is not of particular importance to me, now that you've brought it up...

[Curious.George]

Re: Looking for a NAS

Ok, so what is some of your storage hardware and software that runs on it ? I'm just collecting ideas at this point

For the most part, we use headless USFF boxes with an external USB2 (or 3) drive operated as "appliances" (no display, no keyboard, no mouse, no "apps"). Some drives are just consumer "external disk drives" while others are bare drives installed in a USB SATA dock. Previously, I favored Dell Optiplex 160's for their size and internal power supply. Though we've been replacing them with HP t610's (the 160's were limited to USB2 ports while the t610's also support USB3). The advantage they each have is small size, low power (so I can leave it running while the disk spins down) and fanless operation.

Each is ~1.6GHz dual core, 4GB ram -- which is enough for what they're tasked with doing. But, limited to two USB devices (we seldom mount more than ONE at a time, so not a real issue -- our drives are 250G to 4TB, depending on what we've got stored on them; if you want to access the MP3 archive, then mount the MP3 drive!)

When I need to mount many drives, I use an old PowerEdge 840 (3.3GHz quad core, 4G ram). It has a 4 slot "drive cage" so I can pick 4 (removable) SAS/SATA drives and stuff them inside and move files between them without having to busy the network (drives tend to move about 100MB/s so a single drive can saturate a Gbe link). This is where I typically make (and check) backup copies of drives. If I am highly motivated, I can swap out the SAS/SATA drive cage for an SCA one. Or, rely on an external SCSI enclosure to access content that's still residing on singleton SCSI drives. It also supports USB3 so I can do "local copies" between external USB drives and the "internal" sled-mounted drives.

Because you can't often mix-and-match SATA and SAS drives in a drive cage/shelf, I've chosen to handle my SAS drives in a T420 (8 slots). This ensures that I don't "accidentally" install a SAS drive alongside a SATA one (remember, no displays on these appliances so you never "see" any warnings issued as the box boots up!)

If I need to mount dozens of drives, then I have a set of four 15-drive shelfs that I can bring on-line (typically for audits of the database that keeps track of where every file is located -- I don't have to stand by and reload new drives each time one is finished).

The diskless boxes PXE-boot a custom NetBSD kernel which provides support for the various filesystems (typically just FFS and 9660 -- though I can also support various MS/Linux filesystems if I encounter a medium that happens to use one). It also provides the protocols that I'll use to access the files (I prefer FTP over SMB as it is faster when your concern is TRANSFERING a file, instead of "sharing" it). This lets me "serve" the files in any number of ways: FTP/FXP, SCP, rsync, HTTP, NFS, TFTP, etc. So, different clients can take advantage of the filestore without having to support a particular protocol.

Because each is really a "computer" instead of a dedicated "appliance", I can TELNET/SSH into a box if I need to do something "exotic" that the normal protocols don't efficiently support over-the-wire (e.g., symlinking portions of the file hierarchy). And, can have other services in place like syslogd (dentralized reporting of errors to another server), NTP (to ensure timestamps on files are consistent with those on other boxes running elsewhere on the network), etc.

Someone else might opt to build a similar setup using Linux (or even Windows -- though I'm not sure you can PXE-boot Windows to run diskless). I opt for NetBSD as I've been hacking it for 25+ years, now, so can maintain my own systems without relying on others to fix the bugs or tweek the behavior that may be of concern to me.

[I used to use NTFS-formatted drives but found too many instances where the limitations of things like Explorer made whole portions of the drives inaccessible. Plus, the lack of case sensitivity in Explorer would "hide" certain files that are common in my hierarchies]

And, because most boxes are PXE-booting, I can add another "appliance" just by powering it up and waiting for the kernel to be transfered over the network.

Finally, because there's nothing magical about the media/filesystems I'm using, I can pick up a disk and carry it over to another machine and access it just as "local storage" if the "NAS" dies. Similarly, because there's nothing magical EXPECTED of the media, I can mount a variety of media on the NAS and "serve" that content up on the wire. So, I can access drives having "odd" interfaces (eSATA, FC-AL, SCSI) from machines that don't have those interfaces. Or, media for which "local" drives aren't available (WORM, ORB).

[Dannyx]

Re: Looking for a NAS

Ok, so what is some of your storage hardware and software that runs on it ? I'm just collecting ideas at this point

[Curious.George]

Re: Looking for a NAS

I access the shared files in the explorer shell of Windows directly, so these pictures behave exactly as if they were on the local machine, which is a plus - whichever app wants to open them does so directly without any need for intermediary steps, which is especially handy for someone older without the knowledge/patience to navigate a PC.

We use ours as a scrapbook, of sorts (building on the "precious memories" analogy). Like a scrapbook, it's not instantly accessible but, rather, "we know where to go to find it".

E.g., I archive VMDKs for each project so I can revisit that particular toolset when a client requests a change; no need to tie up a machine "ready and waiting" if he may not need further work done.

My other half takes several thousand photographs each year. Few of these need to remain on her computer (lately, she's been keeping those of interest on a tablet for portability). But, the rest don't want to be discarded.

So, they are moved into the archive with their thumbnails (and keywords) catalogued in a manageably sized database (she uses ThumbsPlus for this). In that way, she can review every photo she's taken (visually as well as by keyword) and select those that she wants to "retrieve" from the archive.

I use a similar technique for my (millions!) of clip-art images which would otherwise be ridiculous to keep on "local" media.

[Dannyx]

Re: Looking for a NAS

if you were just "storing your precious memories", there, then you could (likely) afford to retrieve COPIES of them onto a local filesystem from which you could access (and modify) their content. If you make changes, you could store their updated contents back onto the "storage device" as complete files.

If, however, you expect to see thumbnails created, then the software that is doing that will expect to have direct access to the "files" -- a "network accessible filesystem".

"Files accessible over the network" can be accessed with different/faster protocols (e.g., FTP). But, they don't appear to the software that wants to examine them as files until a local copy has been created. It's "all or nothing" whereas a "network accessible filesystem" would let that software poke into any of those files at any point (offset) WITHOUT requiring the entire file to be copied to local media!.

I access the shared files in the explorer shell of Windows directly, so these pictures behave exactly as if they were on the local machine, which is a plus - whichever app wants to open them does so directly without any need for intermediary steps, which is especially handy for someone older without the knowledge/patience to navigate a PC.

[Curious.George]

Re: Looking for a NAS

The network: the network is a home network ran by a bog standard Dlink router (DIR825) and 99% of clients consist of wireless devices, namely a couple of laptops, so speed is fairly irrelevant at this early stage, though I wish stuff could load faster off of it ! Takes a fair bit of time for explorer to generate the thumbnails and arrange the photos by date like I have it set up and also takes a few seconds for a picture to actually open !

What you're implying, here, is significant. You want a "network accessible filesystem"; not just "files accessible over the network". There's a big (and significant) difference between the two.

E.g., if you were just "storing your precious memories", there, then you could (likely) afford to retrieve COPIES of them onto a local filesystem from which you could access (and modify) their content. If you make changes, you could store their updated contents back onto the "storage device" as complete files.

If, however, you expect to see thumbnails created, then the software that is doing that will expect to have direct access to the "files" -- a "network accessible filesystem".

"Files accessible over the network" can be accessed with different/faster protocols (e.g., FTP). But, they don't appear to the software that wants to examine them as files until a local copy has been created. It's "all or nothing" whereas a "network accessible filesystem" would let that software poke into any of those files at any point (offset) WITHOUT requiring the entire file to be copied to local media!

Like I said, we store our "precious memories" on that NAS, along with kits for programs and games I've gathered over time. It's a convenience to have it tucked away in a cupboard, ready whenever you need it and not rely on a machine running all the time, so that's one aspect.

You can get comparable features with something like an rPi and an external drive (or four). The drive will spin down after a period of inactivity thereby reducing its power consumption to nearly zero. The rPi is also very power efficient.

Backups: I do exactly what you guys say: I have a 1TB drive in a USB caddy and I occasionally "sync" the NAS onto that and sometimes to optical disk. It's running low on space too, so installing a larger NAS is only going to accentuate the issue of backups at one point.

More significantly, it's going to accentuate the cost of rescuing "damaged" data!

If my current NAS (the device) were to fail, recovering the data would be a bit of a challenge, since I'm betting a windows machine wouldn't mount the drives. The only workaround would be to have a machine which uses the NTSF or Exfat file systems, or other formats I'm not aware of but are still "universal".

Depending on the NAS, you may be able to mount the drive on a UN*X machine or access it with specialized Windows tools. But, this assumes you've already got those tools in place and are comfortable using them!

This goes to the heart of my point wrt using standard filesystems and implementing the storage device as "just another disk"; if the machine its tied to dies, unplug the disk and plug it into another machine as "just another disk" (you already KNOW how to access "just another disk", right?!)

When something goes wrong with your NAS/archive, you will probably be very anxious -- possibly panicked! That's not the time you want to be scratching your head trying to remember how to use some special tool to grab the file(s) you need!

My storage array consists of a large number of "bare drives" (typically mounted in sleds) that I can mount into any number of machines; machines that can directly accommodate those sleds or remove the sled and slap the drive in a USB "dock" or install it in a PC. The few remaining drives that are in external enclosures can be retrieved from those enclosures because the enclosures don't add any "magic" to the drives (unlike RAID enclosures).

[Dannyx]

Re: Looking for a NAS

The network: the network is a home network ran by a bog standard Dlink router (DIR825) and 99% of clients consist of wireless devices, namely a couple of laptops, so speed is fairly irrelevant at this early stage, though I wish stuff could load faster off of it ! Takes a fair bit of time for explorer to generate the thumbnails and arrange the photos by date like I have it set up and also takes a few seconds for a picture to actually open ! Like I said, we store our "precious memories" on that NAS, along with kits for programs and games I've gathered over time. It's a convenience to have it tucked away in a cupboard, ready whenever you need it and not rely on a machine running all the time, so that's one aspect.

Backups: I do exactly what you guys say: I have a 1TB drive in a USB caddy and I occasionally "sync" the NAS onto that and sometimes to optical disk. It's running low on space too, so installing a larger NAS is only going to accentuate the issue of backups at one point. If my current NAS (the device) were to fail, recovering the data would be a bit of a challenge, since I'm betting a windows machine wouldn't mount the drives. The only workaround would be to have a machine which uses the NTSF or Exfat file systems, or other formats I'm not aware of but are still "universal".

DIY NAS/router: the 4 ports of the Asus MB would be handy for the router, while the NAS could very well do with that one GE port it's got. The Gigabyte board is a GA-G31M-ES2L BTW

[Curious.George]

Re: Looking for a NAS

That was my second idea: go DIY sort-of. I have a pretty good MB lying around all ready to be put to good use - an Asus P5Q Premium - whose 16x PCI slots no longer work but the 8x ones do (but that's another story). It's got a Xeon X5460 and 8 gigs of DDR2 RAM on it. I was planning to use that to build a PFSense ROUTER instead, the reason being its 4 onboard NICs...

The multiple NICs won't be of much help on a NAS -- unless you want to serve isolated network segments (the bottleneck will almost always be the disks and disk subsystem, unless you use SSDs)

Another MB I have around is a bog-standard, non-impressive Gigabyte GA...something, with a dual core CPU on it if memory serves, and no RAM yet, so I was thinking of using this one for the NAS instead...

Choice of CPU isn't terribly important as you're not doing much "thinking" on the appliance (though a fair bit of "work" moving all those bytes!). I'd focus on lowest power (assuming you want to keep it "available" most of the time) with a GBe NIC

Spend some time looking at the available software to do this (assuming you want a genuine "appliance" and not just an "exported share"). Pay attention to the choice(s) of filesystems supported as well as the protocols that you'll be able to use to access the media. CIFS is "simple" if you're sure all of your clients (and content) will always be Windows compatible. If you have to deal with other platforms, things can get icky.

Likewise, consider carefully IPv6 vs. IPv4 (some appliances have issues with IPv6 affecting overall performance; you may want/need to disable it!)

Think about how MUCH data you will potentially have hosted on this box and how you'll assure yourself that it is intact (when something alarms you that it may not be). And, how you'll move that data to another "safe haven" if the need arises. (try copying a 4T USB2 drive for an exercise in frustration!)

Ideally, look for something that you can install/port on another machine in the event the "real" machine dies or suffers some other problem that would render the files (temporarily?) inaccessible. I.e., if the files are of value, then the media AND the hardware/software to access it are equally so!

[There's nothing more stressful than trying to capture the contents of a BIG disk when it -- or it's host -- is failing!]

[Curious.George]

Re: Looking for a NAS

No, I asked what I wanted to ask. I'm a big fan of KISS.
You're of course free to bloviate with Dannyx but I struggle to read what you write with your parentheses and quotation marks.

Network "speed" factors very little into the choice of a NAS -- or NAS implementation -- as there are so many other issues in the stack between client and server (as well as the amount of network traffic and the network topology)

"A big hard drive attached to your PC that's shared" requires the sharing PC to be up and running in order to be accessible (the same problem that "shared printers" have). You've not indicated the type of i/f that the "big hard drive" would use -- would the i/f be portable enough to sneakernet to another PC in the event of the first PC being unavailable? It's considerably harder for me to share big FC-AL hard drives attached to my SB2000 with other machines because most can't support that interface (so, a failure of the SB2000 is time consuming and in the critical path if I'm interested in accessing any of that data).

"Another the same size attached to another PC as back up" means all replication has to be done over the network. And, either requires a script (or other utility) that the user must remember to invoke to ensure the two copies are in sync.

For KISS, an external USB drive sporting a file system that all of your clients can directly support is the most straightforward solution as it accommodates the elimination of the network fabric, as well.

The bigger PRACTICAL issues involve the clients that will access the media and the protocols that they will use -- esp if you're working in a Windows world.

Try copying:

\\share\some\really\really\REALLY\long\path\name\to\a\particular\file\that\...\I\would\like\to\retrieve\from\my\network\storage\device\having\the\name\Notes

into:

\some\local\folder\that\is\nested\under\a\user's\private\directory

Then, try:

\\share\Notes: 2019
\\share\ReadMe
\\share\readme

Or, worse: try "expanding" an archive (ZIP, SIT, 7Z, TGZ, RAR, ISO, etc.) that contains these sorts of paths/filenames! Esp as you may not have had control over the creation of that archive!!

[Or, do you just disallow these sorts of things to unpack into the NAS?]

[diif]

Re: Looking for a NAS

A better question might be what types of objects do you wish to access, the protocol(s) you want to use and how often (continuously, hourly, daily, weekly, etc.) -- along with the amount of effort you consider "reasonable" for those accesses.

For "long term"/archival storage, an external USB3/eSATA drive (or four) can be a win (portable enough for you to sneakernet it to whichever host "wants" it).

I've found little diskless machines with external drives to be readily expandable, reasonably low power, easy to maintain, etc.. The fact that you can "recover" from a NAS failure by simply unplugging the drive and mounting it on a "regular PC" is a huge win.

[I've been discarding COTS NASs in favor of DIYs because I don't want to have to keep a "spare" of each just for that time when the "one" decides to misbehave]

No, I asked what I wanted to ask. I'm a big fan of KISS.
You're of course free to bloviate with Dannyx but I struggle to read what you write with your parentheses and quotation marks.

[Curious.George]

Re: Looking for a NAS

What speed network do you have ? Why the need for a NAS and not just a big hard drive attached to your PC that's shared and then another the same size attached to another PC as back up ?

A better question might be what types of objects do you wish to access, the protocol(s) you want to use and how often (continuously, hourly, daily, weekly, etc.) -- along with the amount of effort you consider "reasonable" for those accesses.

For "long term"/archival storage, an external USB3/eSATA drive (or four) can be a win (portable enough for you to sneakernet it to whichever host "wants" it).

I've found little diskless machines with external drives to be readily expandable, reasonably low power, easy to maintain, etc.. The fact that you can "recover" from a NAS failure by simply unplugging the drive and mounting it on a "regular PC" is a huge win.

[I've been discarding COTS NASs in favor of DIYs because I don't want to have to keep a "spare" of each just for that time when the "one" decides to misbehave]

[diif]

Re: Looking for a NAS

What speed network do you have ? Why the need for a NAS and not just a big hard drive attached to your PC that's shared and then another the same size attached to another PC as back up ?

[Dannyx]

Re: Looking for a NAS

I have a few spare Buffalo units available 1 rack mount 4 bay and the rest are 4 bay box type, as well as some Isilon 12 bay rackmount units, if you tell me your budget I can give you a price.

Not sure how we'd handle shipping - the price would probably be a killer, to the point where it would make most sense to buy a new device locally

[brethin]

Re: Looking for a NAS

I have a few spare Buffalo units available 1 rack mount 4 bay and the rest are 4 bay box type, as well as some Isilon 12 bay rackmount units, if you tell me your budget I can give you a price.

[Dannyx]

Re: Looking for a NAS

That was my second idea: go DIY sort-of. I have a pretty good MB lying around all ready to be put to good use - an Asus P5Q Premium - whose 16x PCI slots no longer work but the 8x ones do (but that's another story). It's got a Xeon X5460 and 8 gigs of DDR2 RAM on it. I was planning to use that to build a PFSense ROUTER instead, the reason being its 4 onboard NICs...

Another MB I have around is a bog-standard, non-impressive Gigabyte GA...something, with a dual core CPU on it if memory serves, and no RAM yet, so I was thinking of using this one for the NAS instead...

[Topcat]

Re: Looking for a NAS

I've built 2 NAS's for clients based on this, they've both been stellar!!

https://www.supermicro.com/products/...-5028L-TN2.cfm

[Dannyx]

Good day folks. It seems my little WD 500GB NAS is starting to run rather low on space, plus it's a bit outdated now, so I am currently looking for a "good" NAS...with so many options available, ranging from budget ones to enterprise-grade units, I'd like some recommendations, to see what others have to say.

My only main requirement at this point is that I'd like it to have at least 4 bays (the more the better of course, but the prices also go up significantly and they're already at the top limit of what I'd be willing to pay even with just 4). I've done my research and found a WD EX4100 and a QNAP TS-431P2 that would fit the bill for now. I'm still looking around, so the selection is by no means exhaustive and it's open for suggestions, hence the generic discussion.

I'm aware these do not come with HDDs inside, so those also make a huge impact of performance and longevity and I think I'd be willing to spend the extra buck to go with some WD reds....again, some might say it's a good idea, some might call me a sucker...what do you guys like to use in NAS..sess ? *kiddin'

The usage environment is a small home network, used to store pictures, documents, music and all sorts of other miscellaneous stuff a family has from a couple of client machines, but that doesn't mean I can't have a little fun and go a bit overboard