Badcaps.net Forum
Go Back   Badcaps Forums > General Topics > General Computer Discussion
Register FAQ Calendar Search Today's Posts Mark Forums Read

 
Thread Tools Display Modes
Old 07-05-2019, 11:12 AM   #81
Dannyx
CertifiedAxhole
 
Dannyx's Avatar
 
Join Date: Aug 2016
City & State: Constanta
My Country: Romania
Line Voltage: 230VAC 50Hz
I'm a: Hardcore Geek
Posts: 2,417
Thumbs up Re: Looking for a NAS

Quote:
Originally Posted by diif View Post
Unless you're running a speedy RAID array or SSDs it's pointless as your disc speed is the bottleneck.
Correct, that's why I stopped looking into it and decided to focus on improving "disk" speed...somehow
__________________
Wattevah...
Dannyx is offline   Reply With Quote
Old 07-05-2019, 11:46 AM   #82
diif
Badcaps Veteran
 
Join Date: Feb 2014
City & State: Midlands
My Country: England
I'm a: Professional Tech
Posts: 4,041
Default Re: Looking for a NAS

Quote:
Originally Posted by Dannyx View Post
Correct, that's why I stopped looking into it and decided to focus on improving "disk" speed...somehow
An SSD, even on SATA 2 is about 250MB/s with the bonus of low latency and fast random access.
diif is offline   Reply With Quote
Old 07-05-2019, 12:55 PM   #83
Curious.George
Badcaps Veteran
 
Join Date: Nov 2011
Posts: 1,310
Default Re: Looking for a NAS

Quote:
Originally Posted by Dannyx View Post
Correct, that's why I stopped looking into it and decided to focus on improving "disk" speed...somehow
MEASURE before you start optimizing. Premature Optimization is the bane of the naive.

And, measure based on YOUR expected usage patterns, not some canned benchmark (unless all you plan on using the shared data for is running benchmarks!).

There's a reason I transfer files with FTP and not SMB -- performance (for the usage model that I've adopted)!

For an eyeopener, try copying a huge disk (TB+) with millions of files via SMB (i.e., as if you were making a backup of a volume that was remotely mounted). Then, try making a local copy. Note the difference. Same disk. Same OS. Different protocol stack. (Gee, why is it so much SLOWER??!)

[If you really want to get a feel for how underperforming these protocols are, try mounting a datastore remotely and doing queries on the database that uses it!]

Replace the physical disk with a memory disk and see how the numbers change, in each case.

THEN, decide where you want to spend your time/money.

Last edited by Curious.George; 07-05-2019 at 12:56 PM..
Curious.George is offline   Reply With Quote
Old 07-05-2019, 01:48 PM   #84
Dannyx
CertifiedAxhole
 
Dannyx's Avatar
 
Join Date: Aug 2016
City & State: Constanta
My Country: Romania
Line Voltage: 230VAC 50Hz
I'm a: Hardcore Geek
Posts: 2,417
Default Re: Looking for a NAS

Will have to get my hands on the hardware first...don't quite have an SSD on hand readily available, plus a machine to install it in.

Actually I could do some rough tests this way: laptop's got an SSD in it. Share a folder on it, drop some files in there (music), then access them from a wired station, the theory behind all this BS being that it would be the same scenario but in reverse (client is wired and "NAS" is wireless). Pretty flawed, isn't it ? Storage controller speed is probably another (critical) factor here, so there is that...
Dannyx is offline   Reply With Quote
Old 07-05-2019, 02:55 PM   #85
Curious.George
Badcaps Veteran
 
Join Date: Nov 2011
Posts: 1,310
Default Re: Looking for a NAS

Quote:
Originally Posted by Dannyx View Post
Will have to get my hands on the hardware first...don't quite have an SSD on hand readily available, plus a machine to install it in.
You don't need an SSD to do the testing. Pick a disk, ANY disk. Characterize its performance under various usage scenarios: copying to/from the bit bucket (gives you an idea of the disk's max performance with the least OS involvement); to/from another LOCAL disk; to from another remote disk -- under a variety of different protocols and usage patterns.

In my case, I want to copy entire "objects" across the wire. So, a protocol that supports random access is useless overhead (and software complexity): "give me block #1, then block #2, then block #3, ..."

With the "per block" protocol minimized, then you look at the setup time for each transfer (i.e., specifying the object of interest). The client/server have to sort out permissions issues, locking, etc. BEFORE they can start moving data.

This can be a horrendous performance hit when you're dealing with gobs of tiny objects!

My first optimization was to implement server-side compression. So, I could say "get foldername.tar.gz" and the SERVER would build a tarball of the foldername's contents (accessing its LOCAL disk at high speed), compress them (so fewer bytes to transfer over the wire) and, finally, tranfer them to the client.

The downside of this is there aren't as many tools with intuitive interfaces that can understand gzip'd tarballs. And, the archive format has high startup costs if you're just wanting to take a peek inside.

So, now I build .iso's on the server and mount them on the client to give me finer-grained access to their contents.

Quote:
Actually I could do some rough tests this way: laptop's got an SSD in it. Share a folder on it, drop some files in there (music), then access them from a wired station, the theory behind all this BS being that it would be the same scenario but in reverse (client is wired and "NAS" is wireless). Pretty flawed, isn't it ? Storage controller speed is probably another (critical) factor here, so there is that...
If you want to understand the source(s) of performance hits, you need to look at your "system" (client+server) in small pieces -- so you understand what the effects/costs/contributions of each of those pieces are on your overall performance (e.g., before buying a faster disk, newer NAS, etc.). There are lots of micro decisions that your "system" is making of which you are completely unaware.

[I dislike using Windows machines as test platforms cuz there's too much extra cruft running under the hood that can impact performance.]

For example, the actual dialect of SMB that is being used will vary based on the version of the OS (assuming you're just dealing with Windows) that each (client + server) is running. E.g., Vista or 2K8s to/from any newer Windows version will cause both client and server to use dialect 2.0. For 7even/2K8R2 it's 2.1; W8/2K12s it's 3.0; 8.1/2K12R2 it's 3.02 and W10/server10 it's 3.1. The actual performance you'll see in each combination (client+server) will thusly vary (newer dilects try to add functionality, improve performance, and reduce "anomalous behaviors".

Reading from a particular physical disk will give different performance than writing to it.

Reading from a protocol stack gives different performance than writing to it.

Small files give different "steady state" performance than larger ones (read and write caches fill up!)

You want to be able to figure out WHY you're seeing the performance that you are seeing -- is it because of protocol, hardware, software, fabric, access mode, etc.?

UNLESS (of course) you are happy with the performance you are getting in your "typical usage scenario" (in which case, don't look any further!)

[I wanted to get the most out of my "transfer entire files/folders" usage pattern, hence the effort I put into optimizing that path. The SAN I'm building for my VMs has an entirely different goal so a different optimization pattern.]
Curious.George is offline   Reply With Quote
Old 07-05-2019, 03:42 PM   #86
Curious.George
Badcaps Veteran
 
Join Date: Nov 2011
Posts: 1,310
Default Re: Looking for a NAS

Quote:
Originally Posted by Topcat View Post
I just have a hard time trusting critical data to a SSD for some reason.
Agreed. Too many horror stories of nasty (i.e., data-eating) bugs FROM REPUTABLE MANUFACTURERS (e.g., Samsung, Intel).

I think much of the problem comes from the newness of the technology. Folks keep discovering new ways that the media can falter (e.g., read disturb and write disturb in addition to write wear). And, these change with the choice of underlying flash technology (SLC/MLC/TLC/QLC/etc.). Couple this with shrinking geometries and operating voltages and you're counting individual electrons -- a few dozen to differentiate between adjacent bit states (SSDs are becoming ANALOG devices as the controllers attempt to infer more from the actual "charge" seen at each cell to adjust write/refresh algorithms).

This complicates the design of the FTL. Add to that the incentive to push performance limits causes the higher abstraction levels of the device to be merged into the lower levels of the FTL (to exploit efficiency hacks that span the layer boundaries).

Add to that, legacy usage patterns that may be predicated on the assumption that you're dealing with spinning rust which don't hold true for the newer media type (e.g., I can OVERwrite the SAME DATA countless times to a HDD with little cost to the medium; not true for SSDs!)

Bottom line, the firmware gets more complicated and more prone to errors creeping in. Benign errors just cost you some performance. Other errors cost you *data*!

Quote:
I keep good backups of mine....but unless you dump it with every change (which for me would be daily), there's far greater risk for data loss. I usually do a full dump once a month and depend on the RAID (probably more than I should) in the mean time.
I'm setting up a SAN to support snapshotting in the hope that I can just worry about archiving the snapshots, as needed.

Last edited by Curious.George; 07-05-2019 at 03:54 PM..
Curious.George is offline   Reply With Quote
Old 07-11-2019, 06:54 AM   #87
Dannyx
CertifiedAxhole
 
Dannyx's Avatar
 
Join Date: Aug 2016
City & State: Constanta
My Country: Romania
Line Voltage: 230VAC 50Hz
I'm a: Hardcore Geek
Posts: 2,417
Thumbs up Re: Looking for a NAS

Quote:
Originally Posted by Curious.George View Post
In my case, I want to copy entire "objects" across the wire. So, a protocol that supports random access is useless
Quite the opposite in my case: random access IS what I'm doing here with the music and the photos and all that...
Of course, your method could apply here as well: pull an entire file locally THEN play it, but that would probably take the same amount of time as waiting for the thing to buffer to where I want it to - obviously this time gets longer the longer the file is. A 6h track easily takes 1 minute to buffer towards the end....over WiFi that is, sine a wired connection probably cuts down the time significantly. Sadly, WiFi is the way I roll around the house, so I'll have to deal with it and probably any improvements made to the STORAGE side would be hampered by the WiFi bottleneck anyway.

Have you got any WiFi clients in your setup ?
Dannyx is offline   Reply With Quote
Old 07-11-2019, 11:23 AM   #88
Curious.George
Badcaps Veteran
 
Join Date: Nov 2011
Posts: 1,310
Default Re: Looking for a NAS

Quote:
Originally Posted by Dannyx View Post
Quite the opposite in my case: random access IS what I'm doing here with the music and the photos and all that...
Actually, that's no different from copying entire objects -- you just want to be able to access LOTS of different objects. E.g., drag a "Music" (or "Photos") folder over to local storage.

The sort of "random" access that SMB affords (over FTP) is being able to access (read or write) any part of a file without also accessing the whole file. Like tacking a "P.S." onto the end of a file without having to drag the whole file over to do so.

Quote:
Of course, your method could apply here as well: pull an entire file locally THEN play it, but that would probably take the same amount of time as waiting for the thing to buffer to where I want it to - obviously this time gets longer the longer the file is. A 6h track easily takes 1 minute to buffer towards the end....over WiFi that is, sine a wired connection probably cuts down the time significantly.
Our usage pattern is to create local copies of the things that we need, pulling them off the archive only when our needs "change". E.g., loading a new set of music onto a phone; or new apps onto a computer; etc.. So, the archive lies dormant (powered off) for 95% of the time.

Quote:
Sadly, WiFi is the way I roll around the house, so I'll have to deal with it and probably any improvements made to the STORAGE side would be hampered by the WiFi bottleneck anyway.
... unless you can accommodate local copies of those things. E.g., I've got ~3-4T spinning in each workstation so I can just pull entire (archived) projects onto them and poke around to see what I want/need. As I know there's an "original" residing in the archive, there's no problem with me erasing all or part of what I've copied, locally. That way, I don't have to leave the archive (NAS) running any longer than necessary.

Hence the big appeal of using "archives" (.RAR, .ZIP, .TGZ, .ISO, etc.) in the Archive! Just grab it all and toss what you don't need!

Quote:
Have you got any WiFi clients in your setup ?
I have several (7?) hard-wired AP's, here, but leave them off. There are 25 uncommitted "drops" around the house so you're never more than a ~10' patch cord from a network connection.

Last edited by Curious.George; 07-11-2019 at 11:31 AM..
Curious.George is offline   Reply With Quote
Old 07-11-2019, 12:04 PM   #89
Dannyx
CertifiedAxhole
 
Dannyx's Avatar
 
Join Date: Aug 2016
City & State: Constanta
My Country: Romania
Line Voltage: 230VAC 50Hz
I'm a: Hardcore Geek
Posts: 2,417
Red face Re: Looking for a NAS

I was going to go wired whenever possible, such as when the laptop is on the desk, near a readily available switch, but its network card kicked the bucket a while back and I never got around to fixing it (some broken traces on its board as a result of me using a metal screwdriver to lever off the heatsink which)...so for now I'll have to stick to WiFi...

BTW, 7 APs ? Are we talking about your home here or your business ? That's a lot, unless you've got like a 2 acre household
Dannyx is offline   Reply With Quote
Old 07-11-2019, 12:40 PM   #90
Curious.George
Badcaps Veteran
 
Join Date: Nov 2011
Posts: 1,310
Default Re: Looking for a NAS

Quote:
Originally Posted by Dannyx View Post
BTW, 7 APs ? Are we talking about your home here or your business ? That's a lot, unless you've got like a 2 acre household
I use them to track the physical location of "moving nodes". E.g., turn the WiFi on your phone and I can tell if you're in the living room vs. dining room -- and whether you're seated on the sofa, loveseat or one of the chairs.

[I will be gutting 5 of them and replacing them with BT radios so I can track earpieces instead of "phones"]
Curious.George is offline   Reply With Quote
Old 07-11-2019, 12:52 PM   #91
Dannyx
CertifiedAxhole
 
Dannyx's Avatar
 
Join Date: Aug 2016
City & State: Constanta
My Country: Romania
Line Voltage: 230VAC 50Hz
I'm a: Hardcore Geek
Posts: 2,417
Default Re: Looking for a NAS

Isn't that a bit overkill for your average household ? Not gonna judge, it's your space after all Have you got that many visitors around that they constantly require tracking ? I just find it funny actually...in a geeky positive sort of way
Dannyx is offline   Reply With Quote
Old 07-11-2019, 09:22 PM   #92
Curious.George
Badcaps Veteran
 
Join Date: Nov 2011
Posts: 1,310
Default Re: Looking for a NAS

Quote:
Originally Posted by Dannyx View Post
Isn't that a bit overkill for your average household ? Not gonna judge, it's your space after all Have you got that many visitors around that they constantly require tracking ? I just find it funny actually...in a geeky positive sort of way
It's so the House knows that it should turn OFF the TV in the Living Room when I move into the kitchen -- and, turn ON the TV in the kitchen (tuned to the same source, of course). Or, serve up MY music tastes in the Office when I enter it -- or my other half's when SHE does so!

Etc.

Why should I have to TELL the House what to do?? That;s so 1970's...
Curious.George is offline   Reply With Quote
Old 07-11-2019, 11:52 PM   #93
Dannyx
CertifiedAxhole
 
Dannyx's Avatar
 
Join Date: Aug 2016
City & State: Constanta
My Country: Romania
Line Voltage: 230VAC 50Hz
I'm a: Hardcore Geek
Posts: 2,417
Default Re: Looking for a NAS

So you've got some IoT going on...sounds interesting and tempting to do, though for my apartment of 2 rooms (not counting the kitchen and bathroom) that is slightly superfluous. It may sound like I'm living in some sort of coffin-sized dump, but where I live this represents your typical flat in an apartment block and it's fine for 3 people. 3 and 4 room apartments also exist of course, but the prices are way up there, especially if it's located in a "premium" area close to the city center. I was lucky to inherit this one which was in a pretty central area to begin with, but has since evolved into one of the trendiest places in the city thanks to all the malls built around here....I digress
Dannyx is offline   Reply With Quote
Old 07-12-2019, 10:31 AM   #94
Topcat
The Boss Stooge
 
Topcat's Avatar
 
Join Date: Oct 2003
City & State: Salem, MO
My Country: United States
Line Voltage: 120VAC 60Hz
I'm a: Professional Tech
Posts: 12,337
Default Re: Looking for a NAS

Quote:
Originally Posted by Curious.George View Post

Why should I have to TELL the House what to do?? That;s so 1970's...
Ohh come on man, nothing a "Clapper" wouldn't do!!
__________________
<--- Badcaps.net Founder & Owner

Badcaps.net Services:

Premade Capacitor Kits
Badcaps.net Capacitor Master List


Motherboard Repair Services


If you've come here in search of replacement capacitors or repair services, please use the links above.
----------------------------------------------
Badcaps.net Forum Members Folding Team
http://folding.stanford.edu/
Team : 49813
Join in!!
Team Stats
Topcat is offline   Reply With Quote
Old 07-12-2019, 10:41 AM   #95
Curious.George
Badcaps Veteran
 
Join Date: Nov 2011
Posts: 1,310
Default Re: Looking for a NAS

Quote:
Originally Posted by Topcat View Post
Ohh come on man, nothing a "Clapper" wouldn't do!!
Clapper won't know how to ask the living room TV what content its displaying. Or, whether that "program" is over and there's no need to turn on the kitchen TV to watch it's conclusion.

The problem I see with all of the Alexa/IoT offerings is that they are little more than modern version of "remote control" technologies dating back to the 70's (e.g., BSR/X-10). The "user" still ends up having to do all the thinking.

As most of us have highly "regular" (read: repetitive) lifestyles, why can't the devices learn what we'll want and anticipate this, for us? If I've gone to bed, why can't the garage door close itself (because I forgot?). If I've driven off and left the stove on, why can't it turn itself off? If I'm taking a shower, why can't the washing machine pause its rinse cycle (to not cause an uncomfortable drop in water pressure/temperature)? If its going to rain TOMORROW, why can't the irrigation system adjust its cycle so that it only waters those plants ("zones") that can't wait another day for water?

These are all simple decisions that don't need a human intellect to resolve...
("Oh, I'd better go pause the wash cycle before I step into the shower -- and, hopefully, remember to resume it when I'm done...")

Even coffee pots make a token attempt to address these sorts of issues (letting you set a time at which it will start brewing next morning).
Curious.George is offline   Reply With Quote
Old 07-12-2019, 11:10 AM   #96
Curious.George
Badcaps Veteran
 
Join Date: Nov 2011
Posts: 1,310
Default Re: Looking for a NAS

Quote:
Originally Posted by Dannyx View Post
So you've got some IoT going on...sounds interesting and tempting to do, though for my apartment of 2 rooms (not counting the kitchen and bathroom) that is slightly superfluous.
Size of house isn't a defining issue.

I suspect you have a different sleep/wake schedule on weekends than weekdays? (or, whenever your "off" days are) So, the wake time set on your alarm clock should change to match those needs, right? I.e., alarm clock, at the very least, needs to know what day of week it is to compensate.

Ah, but what about holidays? Should it know the legal holidays that are observed in your region? Any other holidays that your employer might observe?

And, what if you have to deal with inclement weather? I.e., it snowed last night while you were asleep and you'll need an extra 10 minutes to prep the car for travel (or, an hour to clear the driveway)? Should the alarm clock take this into consideration? Or, just snicker at you when you wake to discover you're going to be "late"?

In addition to waking at a different time, do your breakfast habits change based on your schedule for the day (e.g., perhaps skip breakfast when you'd rather have an extra half hour of sleep before heading off to work)? Maybe just a quick cup of coffee on those days?

Do you leave the heat/cooling set at the same temperature regardless of whether the house is occupied? Should it be turned down on the days you'll be heading off to work? Should it be turned UP a few minutes before you are expected to get out of bed on weekdays? Weekends?

Do you, perhaps, try to catch the morning news on workdays, while attending to your toilet? And, maybe, instead, listen to music as you're waking up on those days off?

Do you always tune to the Evening News on TV at a particular scheduled time? Or, have any favorite shows on TV/cable in the evenings?

Do you have to remind yourself when you're running low on toilet paper? Or "milk"? Or beer?

If you start looking at all of the little things you've been coerced into doing to be able to live your life, you'll be amused at just how many things you are keeping track of on a daily basis.

Do you know, this morning, BEFORE heading out to the car, whether or not you'll have to plan on stopping at the gas station before work? Did you REMEMBER, last night, to make a mental note of the state of the tank and your expected needs for fuel, today?

Now, imagine you're older and possibly more forgetful. And/or living alone (and wanting to avoid having to be "cared for" at hundreds of dollars per day). How do the consequences of forgetting any of these little things fare? Are there new things that you've never had to address before (e.g., daily medication needs)?

Machines (computers) are great for doing repetitive things. Yet we insist (have no choice!) on doing them ourselves...
Curious.George is offline   Reply With Quote
Old 07-14-2019, 02:47 AM   #97
Dannyx
CertifiedAxhole
 
Dannyx's Avatar
 
Join Date: Aug 2016
City & State: Constanta
My Country: Romania
Line Voltage: 230VAC 50Hz
I'm a: Hardcore Geek
Posts: 2,417
Default Re: Looking for a NAS

Speaking of WiFi: how do you manage security ? WiFi always seemed like fairly vulnerable thing to me, as just about any security feature a (home) router/AP can offer can be breached in some way: hidden SSID and WiFi passwords mean virtually nothing, then you've got the slightly more advanced MAC filtering which of course can also be breached somehow....stuff like that. Sure, it's more than plenty to keep your average idiot out (not that they'd find anything there other than free WiFi) but still, it's nice to know your network is safe from more than just your average Joes. I myself don't know how to crack a WiFI network, mainly because I haven't looked into it deep enough, but I'm sure it's one of the easiest things to do
Dannyx is offline   Reply With Quote
Old 07-14-2019, 03:38 AM   #98
diif
Badcaps Veteran
 
Join Date: Feb 2014
City & State: Midlands
My Country: England
I'm a: Professional Tech
Posts: 4,041
Default Re: Looking for a NAS

Wpa2-psk aes
diif is offline   Reply With Quote
Old 07-14-2019, 04:54 AM   #99
Curious.George
Badcaps Veteran
 
Join Date: Nov 2011
Posts: 1,310
Default Re: Looking for a NAS

Quote:
Originally Posted by Dannyx View Post
Speaking of WiFi: how do you manage security ? WiFi always seemed like fairly vulnerable thing to me, as just about any security feature a (home) router/AP can offer can be breached in some way: hidden SSID and WiFi passwords mean virtually nothing, then you've got the slightly more advanced MAC filtering which of course can also be breached somehow....stuff like that.
Encrypting traffic is the way to secure your comms. There have been several different standards, over the years. Each eventually falling to some exploit or other. (and, rarely does SOHO kit support the sorts of upgrades that are needed to move to more elaborate encryption schemes).

And, the AP/wireless router itself may have flaws that can be exploited "behind" the encryption -- not to mention "defective users". Google "wireless router vulnerabilities" and you'll quickly see that even encryption with a OTP is laughable!

[With any security mechanism, you have to first identify the assumptions upon which you're relying -- then look at how those can be subverted (cuz the attacker will take a close look at them)]

The more insidious (and usually neglected) problem is that there is no way to secure the (wireless) network from DoS attacks. Or, from leaking "activity" and location information.
Curious.George is offline   Reply With Quote
Old 07-14-2019, 11:47 AM   #100
Dannyx
CertifiedAxhole
 
Dannyx's Avatar
 
Join Date: Aug 2016
City & State: Constanta
My Country: Romania
Line Voltage: 230VAC 50Hz
I'm a: Hardcore Geek
Posts: 2,417
Default Re: Looking for a NAS

Quote:
Originally Posted by diif View Post
Wpa2-psk aes
That's how I have it set up, but I was thinking about the more advanced topics of WiFi security
Dannyx is offline   Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump



Badcaps.net Technical Forums 2003 - 2019
Powered by vBulletin ®
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
All times are GMT -6. The time now is 12:57 PM.
Did you find this forum helpful?