Looking for a NAS

[Dannyx]

Re: Looking for a NAS

Back on the topic of networks, what would the best setup for a VPN be ? I.e. access my home network from a remote location, while still doing it somewhat safely. There were so many times where I really needed something, but couldn't get to it because I wasn't home. I have a general idea of how it works, but not at CCNA levels, of course As you can tell, I'm a bit of a noob and do it by ear, rather than by standard.

[Dannyx]

Re: Looking for a NAS

I found THIS for a relatively low price. Bit of 2 in one there - it could handle both my "daily drive" and an occasional torrent. Looking through that spec sheet, there' one thing that set me back however: look at row no. 11: "Maximum internal storage: 4TB"...I know my current WD NAS can only handle 4TB max, but shouldn't this thing be like a "computer" which can handle however much you throw at it ? What if I have two 4TB drives in there, or a single 12TB one ? Will it simply ignore the extra space or not detect the driver entirely ? Some limitation of the storage controller perhaps. That's quite a big disappointment and it's a good thing I looked through the specs, since I was just about ready to get it...must explain the relatively low price, it was a very good buy

[Curious.George]

Re: Looking for a NAS

Other languages, you have absolutely no choice but to start from absolute scratch and get used to an entirely new set of commands, some of which I found not user-friendly at all, stuff which goes a bit over my head and my patience. Not having an end goal also kills the experience for me - no need to try and cram something if I'm not going to use it. I'll forget it and get no satisfaction whatsoever.

Ignoring ASL's, as obvious, most languages are geared towards a particular type of problem and have operators and constructs that directly support those problems. E.g., SNOBOL for pattern matching, FORTRAN for scientific computing, etc.

Beyond being able to understand how to think "systematically" about a solution (algorithm), you also need to understand how a language might facilitate a solution -- or hinder it.

For example, in C, you have to rely on libraries for damn near everything (I/O, multitasking, communications, etc.). Other languages may recognize some/all of these as inherent aspects of the language (e.g., Erlang, Limbo, etc.)

Arbitrarily picking the wrong language -- e.g., "just because you know it" -- can be a recipe for disaster!

Like using BJTs everywhere when a diode or FET might be more appropriate...

[Dannyx]

Re: Looking for a NAS

Just had another idea, or brain fart as I call it. Hardware aside for now, I previously mentioned how I'm having a bit of a hard time keeping my music organized. What I could use is some sort of interface, like a library, to keep track of all the files, including time codes and possibly even snippets of the parts I enjoyed. The best I could do so far, not having put any major effort into doing this properly, is to just have a simple list of plain text doing just that: name of the file+time code.

Not having anything better to do at work today and constantly wanting to learn something new and become less dumb every day, I came across something interesting: PHP. I found a great tutorial on YT that teaches you the very basics and it seems like something easy enough for your average Joe, that's me , to try and I can see how it may actually have some potential in today's world. Our shop's ticket system, or whatever you call that thing that keeps track of the devices brought in for repair, is done in PHP from what I can assume, so I thought "hey, I could probably do this". As I was watching more and more of that series, which is HERE btw, I started thinking about what some of its practical applications may be, what I can actually use it for. No point in learning something if you don't get to use it, plus you'll very likely forget it almost instantly. This is where my idea of organizing my music files came from: writing my own program to do what I want. It may not actually be possible, of course...

I had a couple of failed attempts at picking up SOME form of programming in the past, thinking it's something that has a lot of potential professionally, but was constantly put off by how much it relies on writing code manually and how complex it can get. That's not to say PHP can't get complex and you still have to write it yourself - this is just scratching the surface - but what made it seem slightly more attractive and made me keep on watching was its compatibility with HTML which I messed around with back when I was a kid in highschool, so it wasn't completely new territory and I recognized some of the concepts shown there. Like moving to a foreign country, yet still finding 1-2 friends there by accident...not that I have any *forever alone* >_> Other languages, you have absolutely no choice but to start from absolute scratch and get used to an entirely new set of commands, some of which I found not user-friendly at all, stuff which goes a bit over my head and my patience. Not having an end goal also kills the experience for me - no need to try and cram something if I'm not going to use it. I'll forget it and get no satisfaction whatsoever.

[Curious.George]

Re: Looking for a NAS

Yes, that's another aspect I need to consider - almost critical: noise ! This thing will be right next to the sleeping space (not a lot of room here !), so it needs to be as quiet as possible, at least as quiet as the current NAS+always-on "torrent" PC, which is the maximum my folks are willing to accept at night

Just put something together and "see how it feels". Then, you'll know what issues/criteria need to be optimized for YOUR usage/environment.

I really liked the Dell FX160's -- small/internal power supply, low power, internal support for a 2.5" SATA (plus a SATA SSD), passive cooling (if you elide the hard disk), wireless, reasonably powerful for "moving bytes" and other non-interactive uses (1.6G Atom w/ 4G). VGA+DVI if you ever need to put a head on it (to troubleshoot). And, USB2 is fast enough to support an external drive (or 4) to saturate your wireless network.

Something like that (or any similar USFF/thin client) with a modest size 2.5 internal drive (even diskless machines can often be coerced into supporting a disk drive) as a starting point. Put OS on internal drive so it is self-booting. Create a spare partition on the balance of the drive to hold your "file share".

See how it works. Tether additional external USB drives to it if you want/need to expand storage.

I use one of these with a 640G drive for my TFTP/FTP/NTP/BOOTP/SMTP/DNS/etc. 24/7/365 box. It sits under my dresser (I don't need to access it so why give it "prime real estate" on TOP of some piece of furniture?). If you didn't notice the little blue LEDs on the front panel, you'd never know it was there!

I keep a collection of them as "spares" when I need a sacrificial machine to play with (and don't want to risk "dirtying" one of my workstations). E.g., to download the data from my sphygmomanometer once every few months to appease the MD.

[I also use them to test out distributed software algorithms where I need many physically separated, "generic" CPUs]

[Dannyx]

Re: Looking for a NAS

They are noisy and usually overkill (performance-wise) for SOHO use.

Yes, that's another aspect I need to consider - almost critical: noise ! This thing will be right next to the sleeping space (not a lot of room here !), so it needs to be as quiet as possible, at least as quiet as the current NAS+always-on "torrent" PC, which is the maximum my folks are willing to accept at night I'll have to work around it somehow....and I don't really know how, other than to get a fanless or very quiet setup. Those have the disadvantage that they likely will not have the suitable casing for neatly storing those drives. Sure, I could just leave them out, all over the place, provided it still has the MOBO to accommodate them, but that's a disaster waiting to happen
A Dell sounds like a good idea and I ran across a couple of them at decent prices. Most of them are rack-mountable....tempting, but not doable ATM due to...you know....not leaving alone and not having the joint to myself PLUS, not exactly well suited for many drives, since that's not what they're for. The "tower" cases seem to be ideal for this.

I find it a tedious chore. But, that's because these are "work" machines and not just "toys"

Chore: true, if you're doing this routinely over and over again for years on end, I CAN imagine it becoming "saturated". It's in the human nature to think what others do is so awesome and cool, but once they actually start doing it (IF they get the chance), they get bored eventually.....just like with everything in life >_> So yeah, imagine this: the boss comes up to me and says "Listen up, you're the new head of the IT department. You're in charge of this whole building"....*that day* WOO, we got a party going on...*1 week later* O_O.....no end in sight.....must...stay....alive...carry on...living ! O_O That's how it goes....

However, I must say I DO find stuff like that rewarding, especially if I deal with the hardware as well: there's nothing more satisfying to me than watching my setup which I've built come to life. I currently do security system installs, and it's kinda the same thing....don't ask how I got to this job from being in IT/electronics, but whatever...can't argue with Big Boss or you're shown the door But I digress

[Curious.George]

Re: Looking for a NAS

... since it would still format the drive in EXT4 which a Windows user like me would have trouble using. Sure, if I REALLY had to, I WOULD eventually geti n there, if I really needed to get the data back. I'd learn Linux overnight or something, but like you very well pointed out, it's unlikely I'll think rationally in that situation

You don't wait until you're in that situation. You figure out how to do it, now, while your data is intact.

I've always laughed at folks who flaunt how "religiously" they back up their machines: "When was the last time you tried to RESTORE something from one of those backups? Do you know what the process is? Do you have the right tools? Are you sure the backups are actually intact/recoverable? When you really need it is a lousy time to discover that you're ill-prepared for the task..."

I'm going through that exercise, now, with my SAN -- even though none of its contents are precious (it's effectively a cache for my ESXi servers withthe golden masters stored elsewhere in my archive).

What I'm hunting for presently is a "server" of some sort. I find it hits most of the key points I'm interested in, or at least points which I THINK are important: a case which can hold multiple drives (preferably removable ones), multiple SATA interfaces to host them, a decent storage controller, and dare I say it, dual Gbe links, though this last one is irrelevant ATM but it's there if needed further down the line. Downsides: BIG, heavy stuff. Not exactly compatible with my closet server, is it ?

You likely DON'T want a real server. They are noisy and usually overkill (performance-wise) for SOHO use. The PE840 that I favor for dealing with my "drive collection" was marketed as a low end server many years ago. IIRC, a single 3.4G Xeon -- way overkill for moving bytes on/off disks. But, the 4 removable drives are a huge asset! Unfortunately, its fan is annoying (and runs even when the machine is "off").

Also, the hardware inside might not be designed for anything other than WindowsServer,

Running a FOSS OS on server hardware is usually a safe bet. There are few "specialty" I/Os (touch screens, wireless adapters, etc.) that might not be supported on more consumer-ish hardware. Stuff learned on a FOSS OS is likely more salvageable as the OS evolves than the equivalent Windows stuff (MS seems to like to change stuff often without a genuine need to do so).

Networking is heaps more fun when you've got tons of devices to mess with, isn't it ?

I find it a tedious chore. But, that's because these are "work" machines and not just "toys" -- so, I need things to be up and running, not in various states of disrepair (or, misconfigured, etc.)

I have ~45 "primary" (not counting machines with multiple i/f's) nodes in my office. So, that means having to come up with 45 names (that I can recall without having to consult a cheat sheet), account names, passwords, etc. Each needs an IP address (I assign static IPs so I can access a machine even if I'm having a problem with my DNS) -- and those all have to be recorded in the DNS. MACs need to be recorded for any boxes that use BOOTP. Each box has to be configured to use my time service (and the time server configured to recognize their accesses). Each box has to know how to tie in to my syslog service (so I can peruse error/warning messages in one place, even for boxes that don't have consoles attached!).

Then there are the "normal" network services that you'd expect -- file sharing, printing, TELNET/ssh, FTP, etc. I have my own mail server so I can leave reminders to my "other selves" whenever they use one of the machines that might not be powered up at the present time... easier than leaving post-it notes on each box with a ToDo list!

It's just a sh*tload of "busywork" that you have to keep on top of every time you add/replace a node or make some other change to how the nodes are used.

It's a "cost of doing business". I liken it to a mechanic spending time cleaning his tools after a job, in preparation for the next. It's not really "rewarding" in itself but, rather, a necessary evil ... if you want to be able to do that next job!

[Dannyx]

Re: Looking for a NAS

My point is that the RAID makes those drives "special" -- you can't just move them to another machine and expect to (easily) access them

Correct. That RAID config is presumably only useful if just one drive fails inside the box and the box still has the other one intact so you haven't lost all your data like that *bam* if it were a single drive - almost like a backup copy. When I set up the box for the very first time and it presented me with the RAID options it can use and a brief explanation for them, I naturally went with the "safest" of the choices, RAID1.

If you simply had two drives in a "NAS" configured as JBOD, then you could pull one or both and the NAS wouldn't claim the array was "broken"

It can do JBOD as well, but it would still not be of much use, since it would still format the drive in EXT4 which a Windows user like me would have trouble using. Sure, if I REALLY had to, I WOULD eventually geti n there, if I really needed to get the data back. I'd learn Linux overnight or something, but like you very well pointed out, it's unlikely I'll think rationally in that situation
Here's the spec sheet of this box while we're at it. That SATAII is pretty MEH, though I imagine the network link maxes out before the interface does, esp. over WiFi, which I keep obsessing over because it constitutes around 98% of my small home network.

On Windows, I use BeyondCompare as a file/folder/volume comparison and synchronization tool. So, I can let it compare my current "local" copy to the original and tell me which files need to be updated on the original. Then, let it do the work of moving the copies over

I use something like that too for the purpose of performing that "manual" backup to an external bare drive that I keep in a closet somewhere. It's called "Free file sync" or something like that. Again, not much research done here - just googled for a piece of software that would compare two locations and tell me what was modified, what is new and what needs to be deleted. That drive ? It WILL eventually fill up, so what I do is I burn stuff to optical disks, which is almost like a swear word in the world of computer geeks, but I found it to be the most reliable over the years and we've discussed this before. I only do this with files which I don't need to alter, or at least not often, like pictures....when would you ever need to review and old picture, much less modify it ? A non-photographer ug-mug like myself only "uses" pics once or twice, if that, then they're "memories" which can go in a digital scrapbook in the form of a disk which I can pop in when needed....IF needed...this is that difference in usage scopes and scenarios you were talking about: to others, this may seem unacceptable, blasphemous, but to me, it's perfectly acceptable and I have done it for years now.

What I'm hunting for presently is a "server" of some sort. I find it hits most of the key points I'm interested in, or at least points which I THINK are important: a case which can hold multiple drives (preferably removable ones), multiple SATA interfaces to host them, a decent storage controller, and dare I say it, dual Gbe links, though this last one is irrelevant ATM but it's there if needed further down the line. Downsides: BIG, heavy stuff. Not exactly compatible with my closet server, is it ? Also, the hardware inside might not be designed for anything other than WindowsServer, which may not be so bad in the long run - I could learn something and it might degenerate into a domain or something enterprisey I always wished I could mess around with.....I think I'll need a bigger house I swear, if I ever manage to move out, I'll make sure to stuff the joint to the brim with tech
Networking is heaps more fun when you've got tons of devices to mess with, isn't it ? Lately I started doing more and more field duty and I'd often have to work at some client's firm and get to see their installs and setups and stuff....which kinda make me want to scream and at the same time makes me realize that if THOSE half-wits could get away with it like THAT, there's no reason I couldn't do it AND do it better at that, but I digress

[Curious.George]

Re: Looking for a NAS

That is how I have it ATM: the two disks in the NAS are in RAID 1 and I also manually back that up to another drive from time to time...

My point is that the RAID makes those drives "special" -- you can't just move them to another machine and expect to (easily) access them (a lot depends on how the vendor implemented the array; some go out of their way to obfuscate things -- presumably to tie you to their product/services).

If you simply had two drives in a "NAS" configured as JBOD, then you could pull one or both and the NAS wouldn't claim the array was "broken".

[I have a modified NAS200 (I installed a largish fan inside it) that I use as a low performance "sharing device" operated in exactly this fashion (see pics)]

Where you see the difference is when a RAID fails or the array is operating in "degraded" mode (i.e., trying to compensate for a failure). You're reliant on the NAS (appliance?) for continued access to those files AS THE NAS IS FAILING.

If, instead, they are just "plain disks", then you can pull them out and figure out how/where is best to get at the remaining data before things go tits up.

Starting to think more and more of "building" a NAS like I previously mentioned, even with Windows. Not sure I'd have the time/patience to mess around with custom OSs that require learning programming and other stuff, unless it's (fairly) straightforward like XigmaNAS APPEARED to be.

Most of the "custom OSs" don't really require any "programming". I have done so, heavily, because I want features that I can't BUY (even "for free").

Most just require some knowledge of how to configure the various services that you want to use. E.g., I can configure Apache to automatically present folders that contain photos as thumbnails. Or, let it automatically expand an archive (ZIP/RAR/ISO/TGZ/etc.) when I click on that file.

The turnkey NAS distributions have tried to make best-guess estimates of what users will likely want -- for a machine that has been repurposed as a NAS appliance. They may miss some things that you want/need and provide other things that you could easily live without!

I build custom kernels because I want them to "fit" the hardware as closely as possible (no unneeded drivers, features or capabilities) because the software (including the kernel) has to run entirely out of RAM -- no "system disk" in my appliances!

I haven't looked into it too deep yet - I just searched for "NAS OS" and this popped up and looks like your average Joe, that's me , can pull it off...Ok, my PC skills are slightly above average, but you get the idea . Only downside is that such a box takes up considerably more space than a NAS box....

That depends on your packaging skills -- and the amount of "stuff" that you have on hand (or easily accessible).

I keep my drives in one of three (interchangeable) configurations:

• bare
• affixed to legacy Dell carrier
• affixed to more modern Dell carrier

Bare drives take up the least space. But, need to either be mounted into a carrier or installed in a dock (or, some other means of getting them "into" a machine).

Most SATA drives I mount in legacy Dell carriers. I have a machine with a small 4 drive cage (PowerEdge 840 -- see pic, bezel removed) that I tend to use to access them -- I stuff one to four drives in the machine and they appear as /0, /1, /2, /3 which I can then access locally (TELNETing into the machine) or remotely (HTTP, FTP, rsync, etc.) over the network.

Because the volumes appear as such generic folders (0, 1, 2, 3), I make a point of creating a file hierarchy that is recognizeable to me once I navigate INTO one of those mount points. So, a drive that is entirely photos won't just have folders that appear as:

• /0/Back Yard
• /0/Mountains
• /0/Repairs

etc. But, rather, as:

• /0/Photos/Back Yard
• /0/Photos/Mountains
• /0/Photos/Repairs

etc. I.e., I will ignore the "/0" when I peruse the volume and treat it as if it was "/Photos/..."

I've a few four slot cages that I'm looking into mounting into a small case (like a Shuttle). This would let me replace that 4 slot PC with a smaller box for the same type of convenience. I could also build different versions -- one with a 4 slot SCA cage, one configured for SAS drives, etc. (see pic)

An alternative would be that "slot" visible in the PE840 photo with the system disk partially exposed (below the optical drive). I've not found any that are suitably robust, though (they all feel cheaply made -- as if you're only RARELY supposed to remove/install disks!

I "store" the carrier-mounted drives in one of two (two copies of everything!) 45-slot DASs (see pic) until needed. If I ever need to access "significant" portions of my archive concurrently (e.g., when verifying catalogs of their entire contents), I can spin up the DASs and look at everything at once (instead of mounting one drive at a time in a dock, etc.)

I've chosen to use the "newer" Dell carriers in my SAN. This helps ensure that I don't mix-and-match drives between the DAS and SAN. The SAN drives are RAID configured (their contents are sourced from the golden masters on the DASs) for availability (not durability!). If I lose the contents of the SAN, I don't actually "lose" anything -- other than the time to copy the golden masters back onto them! So, I'm not too concerned about how the SAN uses the drives to implement the arrays (RAID 10's). Though I will see if any of the existing raid recovery tools that I have will be of use... "just in case"!

I rarely have any "outboard" storage powered up. I have gobs of disk space on each machine so if I am going to need something, I just drag it all onto a workstation (from "wherever") and work with a local COPY of the stuff. If I make changes that I want to save, then I can push THOSE CHANGES back onto that same "wherever".

On Windows, I use BeyondCompare as a file/folder/volume comparison and synchronization tool. So, I can let it compare my current "local" copy to the original and tell me which files need to be updated on the original. Then, let it do the work of moving the copies over.

Similarly, if I want to pull stuff off of "wherever", I let it compare the original to an (empty!) destination folder... in which case, it indicates a need to copy EVERYTHING over! :>

[I really like this program as it is reasonably well designed/implemented. A few warts that I'd prefer didn't exist but I can work around them, for the most part]

UNLESS USB: that setup of yours is not a bad idea, especially for home use.

For the average home user, plug an external USB disk (in a nice plastic case) in and forget dealing with bare drives. This is the approach most of my colleagues have adopted as you can buy large (consumer) USB drives for ~$20/TB in a portable case. All they care about is a few years of history so if the drive dies, later, they just write it off as a "consumable".

I'm just curious how much of a difference a SATA interface makes over the USB 3.0 one you've got there, assuming I get a dock+MOBO that has 3.0. The ones I've got currently don't, so that would be a potentially expensive hurdle to overcome. What drives are those BTW ?

I look for thin clients and other USFF boxes that have USB3 interfaces. They tend to be small and have lower power requirements. Often fanless, as well (sadly, the Lenovo boxes require active cooling -- the t610s that they are replacing were entirely passive). And, as they seem to be falling out of favor, they can usually be acquired "cheap".

The manner in which the USB controller interfaces to the host CPU seems to be the biggest driver of performance. E.g., PCI vs. PCIe vs. other motherboard options will determine how fat the pipe is to the CPU.

For USB2, I always assume ~20MB/s. So, I'll almost never try to copy a large volume using a USB2 interface (1T=14hrs)! OTOH, if I want to copy a few MP3's off of a 2T drive, there's no reason to avoid USB2!

USB3 I assume about 70MB/s. Consumer (external) drives and cheap docks tend to max out at around this rate. So, its possible to copy large-ish volumes -- but still pretty painful (1T=4hr). Using an "internal" SATA interface, I figure 120-150MB/s (1T=2hr) -- so, the network would be the bottleneck in a NAS.

[Imagine how long a RAID has to rely on the drive/array being operational if you're trying to recover its contents! And, RAID has to "do some work" beyond just "moving bytes" off the drive!]

[Dannyx]

Re: Looking for a NAS

You can get RAID 1 without having to invest in a RAID appliance just by storing two copies of the data on two different spindles. If you use generic filesystems (instead of any of the "magic" that NASs often put on their drives), then you have the same amount of protection -- albeit at a bit more effort (cuz YOU have to create the multiple copies of the data)

That is how I have it ATM: the two disks in the NAS are in RAID 1 and I also manually back that up to another drive from time to time...

Starting to think more and more of "building" a NAS like I previously mentioned, even with Windows. Not sure I'd have the time/patience to mess around with custom OSs that require learning programming and other stuff, unless it's (fairly) straightforward like XigmaNAS APPEARED to be. I haven't looked into it too deep yet - I just searched for "NAS OS" and this popped up and looks like your average Joe, that's me , can pull it off...Ok, my PC skills are slightly above average, but you get the idea . Only downside is that such a box takes up considerably more space than a NAS box....UNLESS USB: that setup of yours is not a bad idea, especially for home use. I'm just curious how much of a difference a SATA interface makes over the USB 3.0 one you've got there, assuming I get a dock+MOBO that has 3.0. The ones I've got currently don't, so that would be a potentially expensive hurdle to overcome. What drives are those BTW ?

[Curious.George]

Re: Looking for a NAS

George shows people other options. RAID -- in any flavor -- is typically NOT needed for "normal users" and George's approach (of simple disks with traditional filesystems) is more likely going to net them a higher performing (in terms of data retention) solution than a NAS which may typically NOT let the user just pull a drive and stick it in their PC (to recover files).

"...something simple" Like an external USB disk. RAID 0 is NOT simple any only indicated when you want the performance of striping a disk over two (or more) drives... and are willing to live with the TOTAL loss of ALL your data if any of those drives bites the dust.

Hardly "simple" for "the normal user".

My latest NAS: Lenovo ThinkCenter M92p (2.6GHz i3) pulled from the scrap pile, today, tethered to a 3 drive USB3 dock (stuffed with 2T drives, presently). There's a second USB3 port on the Lenovo so I could add another 3 drive dock, if wanted.

I pulled the 500G disk from the Lenovo (run it diskless so it's easier to maintain and administer) and replaced the 16G RAM with 4G (no need for all that memory); removed the second DP port (it's a headless appliance so no need for a monitor, let alone TWO -- though being able to plug one in at any time to interact DIRECTLY with it is a big win) and the wireless card to save a wee bit of power (access over wired Gbe connection).

Disks are UFS formatted so I can pull them out of the dock and install them into another machine -- or, into another dock tethered to a different machine (or NAS!) and access them, there, immediately without any "special tools".

PXE boot a custom NetBSD kernel with an NFS mounted userland that provides HTTP, FTP/FXP, rsync, Samba, NFS, SCP, etc. access to the mounted drives. Errors logged to a syslogd service on another 24/7/365 machine so I can track performance without needing a console on the NAS. TELNETd and SSHd for remote administrative access.

(Of course, all of this could reside on an internal disk -- like the one I removed -- if you don't mind having to update the software running IN the NAS to make changes to its operation. Doing it my way means I can take the next M92p and turn it into a second NAS just by adding a line or two to a configuration file)

If I want to copy (or compare) a large portion of a drive to another, I can let the "NAS" do it, internally (instead of having to pull the files onto a separate client OVER THE NETWORK and then push them back onto the destination drive AGAIN OVER THE NETWORK on the NAS).

Because it's really a COMPUTER not just a (closed!) NAS!

If I want to do a patrol read, I just have a script start walking through the mounted filesystems reading "everything" in an attempt to discover disk errors (in practice, I direct the order of files being examined based on how often I want each to be checked and records of when they were last checked)

If I want an inventory of everything on the drive(s), I just walk the filesystem fstat(2)-ing everything I encounter.

If I want to verify the contents of those objects are unchanged, I just compute their current hash values and compare them to stored hash values.

If the files I want don't happen to be contained in this 6T of rust, I can just pull a drive and plug in another!

Of course, with work, you could probably coax a Windows OS to provide similar functionality. Or, a Linux box.

When I stumble across a smaller/faster USFF box (or thin client), I can swap out the Lenovo for whatever new toy comes along (the Lenovos are currently replacing t610s... which replaced FX160s... which replaced CA17's... which...)

[diif]

Re: Looking for a NAS

It's a Dlink DIR 825...bog standard stuff, but I'm planning to upgrade....somehow...not sure what I should go for. I was planning to build a PFSense router and add an AP to it...somehow The "internet gurus" in our company seem to like Ubiquity APs....

I only hear good things about their stuff from fellow techs, enterprise features, reliable, no recurring software fees and a good price.

[Curious.George]

Re: Looking for a NAS

I'm sensing this will degenerate into some argument

Back to the issue at hand: RAID 0....isn't that "risky business" ? I was thinking maybe put two disks in RAID 0, another two also in RAID 0 and THEN the two sets in RAID 1...if such thing can even be done or is a remotely useful...

[Google RAID-on-RAID or RAID 10, RAID 50, 60, etc.]

RAID 0 is just plain stupid. RAID 0 puts 1/N of the data on each of N disks (N being the number of disks you have in the group -- typically 2 but easily more). This allows data to come off the SET of disks at a much higher aggregate rate than it would off a single disk.

A low performing consumer disk will deliver ~40MB/s. That's 30% of a Gbe link. A decent consumer disk will deliver ~70MB/s -- now you're at ~60% of a Gbe link (we're assuming the client(s) on that link can suck up the data at that rate.

Put 2 disks in a RAID 0 configuration and your disks will outperform your network connection. So, you're wasting performance.

And, in return, you are putting all of your data at risk for a single drive failure (regardless of how many drives you have in the group).

RAID 1 is the simplest (in terms of complexity and cognitive effort) array to safeguard your data. It duplicates data on multiple spindles. So, your performance is that of a single disk (typically a smidgeon less). But, if a single drive fails AND THE RAID HARDWARE/SOFTWARE IS PROACTIVE, you don't lose any data (unless subsequent failures strike the remaining drive(s)).

You can get RAID 1 without having to invest in a RAID appliance just by storing two copies of the data on two different spindles. If you use generic filesystems (instead of any of the "magic" that NASs often put on their drives), then you have the same amount of protection -- albeit at a bit more effort (cuz YOU have to create the multiple copies of the data).

E.g., two external disks on a scrap PC or in a NAS appliance that can be configured to treat the drives as JBOD. (Some SOHO NASs have USB ports so you can attach external drives and get exactly this sort of configuration -- so you can later move the drives to "regular" PC's just by unplugging them)

Generic filesystems means you aren't tied to the NAS appliance if you want to read/recover data (e.g., if the NAS shits the bed). And, you don't need special "RAID recovery software" to try to access the contents of the drives when the shit DOES hit the fan.

See Capleaker's thread on that RAID failure. How long will the data be inaccessible? How much of it will never be recovered? How much work/stress will be involved?? (How likely is a "normal user" to be able to pull off that sort of a recovery effort??)

[Curious.George]

Re: Looking for a NAS

Back to the ops needs, not what George has, cause he isnt the norma!l user and his help is always overblown and unhelpful for normal users.

George shows people other options. RAID -- in any flavor -- is typically NOT needed for "normal users" and George's approach (of simple disks with traditional filesystems) is morel likely going to net them a higher performing (in terms of data retention) solution than a NAS which may typically NOT let the user just pull a drive and stick it in their PC (to recover files).

If you have 1k eitherhet then any set of spinners in a raid 0 will fill your needs if the raid server isn't some seriously pata server or something. unless your trying to server like 10+ clients at the same time, all you need is a something simple.

"...something simple" Like an external USB disk. RAID 0 is NOT simple any only indicated when you want the performance of striping a disk over two (or more) drives... and are willing to live with the TOTAL loss of ALL your data if any of those drives bites the dust.

Hardly "simple" for "the normal user".

Dont let George make it seem like you need a data center just because thats he does for some ungod known reason in his house of all places. lol

George runs a business and has 40 years of historical data (designs and the tools used to create them) to maintain in his archive. I suspect your "history" is limited to a "90 day warranty"...

Capleaks seems to just be the latest RAID victim. Wonder how much of that 12T? will be lost -- or, how much time/money it will take to recover it.

[George hasn't lost any data in 40 years. George has undoubtedly spent far less time or money than Capleaks on his storage array.]

[Dannyx]

Re: Looking for a NAS

I'm sensing this will degenerate into some argument

Back to the issue at hand: RAID 0....isn't that "risky business" ? I was thinking maybe put two disks in RAID 0, another two also in RAID 0 and THEN the two sets in RAID 1...if such thing can even be done or is a remotely useful...

[brethin]

Re: Looking for a NAS

Back to the ops needs, not what George has, cause he isnt the norma!l user and his help is always overblown and unhelpful for normal users.

If you have 1k eitherhet then any set of spinners in a raid 0 will fill your needs if the raid server isn't some seriously pata server or something. unless your trying to server like 10+ clients at the same time, all you need is a something simple. Dont let George make it seem like you need a data center just because thats he does for some ungod known reason in his house of all places. lol

[Dannyx]

Re: Looking for a NAS

It's a Dlink DIR 825...bog standard stuff, but I'm planning to upgrade....somehow...not sure what I should go for. I was planning to build a PFSense router and add an AP to it...somehow The "internet gurus" in our company seem to like Ubiquity APs....

[diif]

Re: Looking for a NAS

That's how I have it set up, but I was thinking about the more advanced topics of WiFi security

What wireless AP/router are you using ?

[Dannyx]

Re: Looking for a NAS

Wpa2-psk aes

That's how I have it set up, but I was thinking about the more advanced topics of WiFi security

[Curious.George]

Re: Looking for a NAS

Speaking of WiFi: how do you manage security ? WiFi always seemed like fairly vulnerable thing to me, as just about any security feature a (home) router/AP can offer can be breached in some way: hidden SSID and WiFi passwords mean virtually nothing, then you've got the slightly more advanced MAC filtering which of course can also be breached somehow....stuff like that.

Encrypting traffic is the way to secure your comms. There have been several different standards, over the years. Each eventually falling to some exploit or other. (and, rarely does SOHO kit support the sorts of upgrades that are needed to move to more elaborate encryption schemes).

And, the AP/wireless router itself may have flaws that can be exploited "behind" the encryption -- not to mention "defective users". Google "wireless router vulnerabilities" and you'll quickly see that even encryption with a OTP is laughable!

[With any security mechanism, you have to first identify the assumptions upon which you're relying -- then look at how those can be subverted (cuz the attacker will take a close look at them)]

The more insidious (and usually neglected) problem is that there is no way to secure the (wireless) network from DoS attacks. Or, from leaking "activity" and location information.