Re: well I lost everything on my server
(sigh) Sad little person. You've obviously not considered the consequences of your decisions. Fear not, life will teach you these lessons! 
You'll remember this discussion and be even angrier at me when you realize the prophecy! 
Ask yourself how you're going to increase your storage capacity: copy everything off the ZFS onto ???; tear down your vdevs, add another drive and then rebuild everything and copy everything back? Or, replace drives, one at a time in each vdev with larger drives, resilver and then then let the pool expand? Discard the old (now too small) drives? Add a second vdev (3 more drives) -- if the hardware can support it? Add a second box with a different configuration (components of which will be useless as spares for the first box; and vice versa).
You may not have 100T now. But, once you start getting used to having "unlimited storage" you'll discover that you use it, fast. With ZFS/RAID, all of the data is spinning at any given time. So, it's all AT RISK. A careless keystroke, a catastrophic hardware failure, a bug in the software, etc.
(So, just keep duplicates of EVERY box that you have as insurance!
)
I add new drives just by plugging them in (unplugging something that isn't CURRENTLY being used). I keep 1, 2 or 30 copies of files as I see fit. They can reside on the same volumes or on N different ones -- in any combination. They can have different names and reside in different directories across volumes -- beyond even the capacity of symlinks to map equivalences!
I can mail a medium to a client and he can read/write it at his leisure. When it comes back to me, I can see what (if anything) has been changed. I can store any portion of my archive offsite on as many volumes as I see fit (including just one). A client can mail me a volume and I can import it to my archive "as is" -- so, when he refers to "/projects/2010/summer/proposals/blah" I'll be able to directly and unambiguously access it.
I can access it via Solaris, Windows (XP, Vista, 7, 8, 10 -- even 95/98 with USB support), Linux, MacOSX -- or even a thin client with just 100MB of RAM (and no display!). I can upgrade any of these OS's and never put more than one volume at risk (to prove the integrity of the OS upgrade).
I can browse my entire repository's namespace without mounting ANY volumes. I can create a new volume drawing files from any number of volumes. I can create volumes on R/O media. I can do this on a laptop without AC mains being available.
I can FREELY incorporate volumes of different types, technologies and interfaces (SCA/FC-AL/SCSI/SATA/PATA/floppy/memory card/etc.). I can export an NFS/SMB share and treat it as part of the archive -- verifying its integrity or automatically "reinitializing it" (mark it as corrupt) with the "backup originals" from the rest of the archive.
I don't need special tools to do any of them (other than some SQL scripts and the ability to mount volumes in whatever way the host OS desires). So, I don't have to worry about updates to those tools "breaking things" or putting the entire archive at risk.
You can do all of those things with ZFS, right?
In your professional capacity
, address the "hypothetical" client with 100T of storage doubling every 3-4 years. How should he ensure he can access old tax records (in the UNLIKELY event of an audit), old sales histories (when proposing a new product), old development costs (when budgeting the development effort for that new product), etc. Be sure to tell him what sort of equipment he should expect to buy and what sort of staff (and labor costs) he should be prepared to encounter in order to maintain that data. Prepare a budget of the hardware maintenance costs for the foreseeable future (how long before those drives fail? How much money and time to replace?
I'll gladly present your recommendations to MY clients -- as EXPERT ADVICE. So, be sure to reflect all the costs/benefits in your proposal.
Or, are you a one-trick pony? Would you have advocated building an addition on the corporate office to hold more FILE CABINETS to keep ALL of the companies records "accessible" -- despite the age or value thereof -- instead of moving old stuff to "secondary storage"? Sure sounds like it!
To mimic your comments, based on how evasive you have been over real issues (you haven't even disclosed the size of your array, number of objects it contains, the hardware its running on, your costs to implement/maintain, etc.) and other wild claims, I'm sorry to say that I'm leaning more toward you being a simple ZFS fanboi rather than credible.
[or, having sorely limited experience "in the wild"... ever work for a company with 10 employees? 100? 1000? 100000? If you have, you'll know that the way problems are solved varies greatly, as do the resources available to throw at problems! Hint: I've worked at or for all of the above and had to come up with solutions that "made sense" to each of them]
I'll respond when you've proposed a solution that I'll gladly pass by real clients for their unbiased opinions. I'm sure the big companies with systems in place will be EAGER (not!) to adopt your solution. As will the companies with no IT departments.
Originally posted by sofTest
View Post
You'll remember this discussion and be even angrier at me when you realize the prophecy! Ask yourself how you're going to increase your storage capacity: copy everything off the ZFS onto ???; tear down your vdevs, add another drive and then rebuild everything and copy everything back? Or, replace drives, one at a time in each vdev with larger drives, resilver and then then let the pool expand? Discard the old (now too small) drives? Add a second vdev (3 more drives) -- if the hardware can support it? Add a second box with a different configuration (components of which will be useless as spares for the first box; and vice versa).
You may not have 100T now. But, once you start getting used to having "unlimited storage" you'll discover that you use it, fast. With ZFS/RAID, all of the data is spinning at any given time. So, it's all AT RISK. A careless keystroke, a catastrophic hardware failure, a bug in the software, etc.
(So, just keep duplicates of EVERY box that you have as insurance!
)I add new drives just by plugging them in (unplugging something that isn't CURRENTLY being used). I keep 1, 2 or 30 copies of files as I see fit. They can reside on the same volumes or on N different ones -- in any combination. They can have different names and reside in different directories across volumes -- beyond even the capacity of symlinks to map equivalences!
I can mail a medium to a client and he can read/write it at his leisure. When it comes back to me, I can see what (if anything) has been changed. I can store any portion of my archive offsite on as many volumes as I see fit (including just one). A client can mail me a volume and I can import it to my archive "as is" -- so, when he refers to "/projects/2010/summer/proposals/blah" I'll be able to directly and unambiguously access it.
I can access it via Solaris, Windows (XP, Vista, 7, 8, 10 -- even 95/98 with USB support), Linux, MacOSX -- or even a thin client with just 100MB of RAM (and no display!). I can upgrade any of these OS's and never put more than one volume at risk (to prove the integrity of the OS upgrade).
I can browse my entire repository's namespace without mounting ANY volumes. I can create a new volume drawing files from any number of volumes. I can create volumes on R/O media. I can do this on a laptop without AC mains being available.
I can FREELY incorporate volumes of different types, technologies and interfaces (SCA/FC-AL/SCSI/SATA/PATA/floppy/memory card/etc.). I can export an NFS/SMB share and treat it as part of the archive -- verifying its integrity or automatically "reinitializing it" (mark it as corrupt) with the "backup originals" from the rest of the archive.
I don't need special tools to do any of them (other than some SQL scripts and the ability to mount volumes in whatever way the host OS desires). So, I don't have to worry about updates to those tools "breaking things" or putting the entire archive at risk.
You can do all of those things with ZFS, right?
In your professional capacity
, address the "hypothetical" client with 100T of storage doubling every 3-4 years. How should he ensure he can access old tax records (in the UNLIKELY event of an audit), old sales histories (when proposing a new product), old development costs (when budgeting the development effort for that new product), etc. Be sure to tell him what sort of equipment he should expect to buy and what sort of staff (and labor costs) he should be prepared to encounter in order to maintain that data. Prepare a budget of the hardware maintenance costs for the foreseeable future (how long before those drives fail? How much money and time to replace?I'll gladly present your recommendations to MY clients -- as EXPERT ADVICE. So, be sure to reflect all the costs/benefits in your proposal.
Or, are you a one-trick pony? Would you have advocated building an addition on the corporate office to hold more FILE CABINETS to keep ALL of the companies records "accessible" -- despite the age or value thereof -- instead of moving old stuff to "secondary storage"? Sure sounds like it!
To mimic your comments, based on how evasive you have been over real issues (you haven't even disclosed the size of your array, number of objects it contains, the hardware its running on, your costs to implement/maintain, etc.) and other wild claims, I'm sorry to say that I'm leaning more toward you being a simple ZFS fanboi rather than credible.
[or, having sorely limited experience "in the wild"... ever work for a company with 10 employees? 100? 1000? 100000? If you have, you'll know that the way problems are solved varies greatly, as do the resources available to throw at problems! Hint: I've worked at or for all of the above and had to come up with solutions that "made sense" to each of them]
I'll respond when you've proposed a solution that I'll gladly pass by real clients for their unbiased opinions. I'm sure the big companies with systems in place will be EAGER (not!) to adopt your solution. As will the companies with no IT departments.
).
Comment