Badcaps.net Forum
Go Back   Badcaps Forums > General Topics > General Computer Discussion
Register FAQ Calendar Search Today's Posts Mark Forums Read

 
Thread Tools Display Modes
Old 02-13-2019, 07:04 PM   #1
Curious.George
Badcaps Veteran
 
Join Date: Nov 2011
Posts: 1,304
Default Disk conidence test while wiping

I've married an expert system to my disk wiping software. It looks at "data" from the current drive, coupled with historical data for previously encountered drives of that make/model (because we encounter drives in large, identical batches).

Initially, the AI gives me an assessment of whether or not the wipe is exhaustive or runs the risk of leaving data behind (which dictates physical destruction is required).

[I don't want to have a separate "disk test" activity -- esp as the disk will be hammered on more or less continuously during the wipe... problems SHOULD be pretty obvious!]

But, it is also intended to assess the drive's "future reliability prospects" -- i.e., should it be returned to service after it has been wiped or discarded.

I use lots of "macro" data as well as "micro" data in forming this decision. (e.g., macro: average wipe rate; micro: number of retries for each write operation).

Most operating systems hide all of the "micro" data from the application/user. E.g., an OS will repeatedly attempt to reissue a "write" until the drive processes it -- before resorting to complaining to the application that "write failed".

I'm looking to understand what I can reasonably expect from a "working" drive at this level of detail. E.g., why would a good drive ever "fail" a write operation... but succeed when it is retried? (perhaps if a sector remap operation was initiated within the drive as a result of the commanded write operation?)

I know I can't blindly rely on the "time" required for an operation to complete as I have to expect recals, remaps, autospin-ups, etc. to alter these figures...

Last edited by Curious.George; 02-13-2019 at 07:05 PM..
Curious.George is offline   Reply With Quote
Old 02-14-2019, 08:01 PM   #2
brethin
Badcaps Veteran
 
Join Date: Dec 2008
City & State: Owensboro, KY.
My Country: USA
Line Voltage: 120VAC 60Hz
I'm a: Professional Tech
Posts: 1,708
Default Re: Disk conidence test while wiping

reinventing the wheel? plenty of good wiping software out there and unless you have DOD drives no need to do more than a full write 1 then write 0, takes 2 passes and done.
brethin is offline   Reply With Quote
Old 02-14-2019, 10:48 PM   #3
Curious.George
Badcaps Veteran
 
Join Date: Nov 2011
Posts: 1,304
Default Re: Disk conidence test while wiping

Quote:
Originally Posted by brethin View Post
reinventing the wheel? plenty of good wiping software out there and unless you have DOD drives no need to do more than a full write 1 then write 0, takes 2 passes and done.
Most software sits atop the OS -- so you don't actually see what is happening at the disk interface level (the OS tries to make operations work, not report potential "issues" with the device).

Depending on the OS/BIOS, you may not be able to remove the DCO & HPA (assuming the wiping software is even aware of them!)

Nor does the software have any smarts to determine WHICH drives are not worth the trouble to wipe and reuse (because the software doesn't track the past performance of drives).

[Hence the rationale for the questions in my post]

And, of course, that software is not likely to print QR/bar-coded labels for the drives and log them into "inventory" -- along with the data observed in the wiping process.

Finally, wiping MANY drives simultaneously requires a lot more "hardware" (MIPS) when you have the OS's bloat in the middle. E.g., try wiping 60 drives concurrently and let me know what your aggregate disk bandwidth is. Then, divide by number of spindles and discover just how much time was wasted waiting for the software (the DISKS should be the limiting factor).
Curious.George is offline   Reply With Quote
Old 04-01-2019, 03:21 PM   #4
Curious.George
Badcaps Veteran
 
Join Date: Nov 2011
Posts: 1,304
Default Re: Disk conidence test while wiping

Finally assembling the hardware for this. Installed 7 disk shelfs, today (5 to wipe and 2 for RDBMS). Plus keyboard/monitor and a small switch.

Still to add: two servers (one to wipe disks, the other for the RDBMS), a pair of rack-mountable UPSs and firewall appliance. And, a "smart" PDU.
Attached Images
File Type: jpg Rack.jpg (650.2 KB, 5 views)
Curious.George is offline   Reply With Quote
Old 04-11-2019, 05:36 PM   #5
ron350
Badcaps Veteran
 
Join Date: Feb 2014
City & State: AL
My Country: USA
I'm a: Knowledge Seeker
Posts: 212
Default Re: Disk conidence test while wiping

CG when you get this working will the company or charity sell on ebay?
Thanks Ron
ron350 is offline   Reply With Quote
Old 04-12-2019, 01:18 PM   #6
Curious.George
Badcaps Veteran
 
Join Date: Nov 2011
Posts: 1,304
Default Re: Disk conidence test while wiping

Quote:
Originally Posted by ron350 View Post
CG when you get this working will the company or charity sell on ebay?
Thanks Ron
The initial provider (there are several other charities asking for copies of the machine) presently supplies computers to underprivileged clients. The computers are typically donated by businesses (periodic upgrade cycles where they replace every machine in their organization). One condition of the donation(s) is that the machines be purged of any data they may contain (even removing asset tags and other information stored in NVRAM). A client should never be able to tell where the machine came from. And, the donor should never have to worry that their "data" may be leaking out.

Wiping the disks also protects the CLIENT from any consequences of "residual data" that may be left on the machine from its previous owner/user.

Doing the disks individually is just too time consuming -- roughly an hour for a 250G-500G drive (determined largely by the write speed of the drive). My goal is to be able to wipe 120T per "shift" (i.e., 120 1T drives, or 240 500G drives, or 480 250G drives, or combinations thereof). This is a monotonous task that is easily prone to error ("Hmmm... did I wipe this drive, already? Or, was it THIS one? To know for sure, I'll have to READ the entire drive to verify its contents -- which takes just as long as WIPING it!")

This also avoids the problems inherent in naive/ad hoc wiping solutions (e.g., DCO and HPA). It can be configured to be as pedantic as we deem appropriate (e.g., discard drives with remapped sectors; follow-up verify pass, etc.). It also provides an indication of the "quality" of the drive by noting how well it performed during that process (how fast is it, does it throw any errors, etc.)

It incorporates a database that will, ultimately, let it also act to install disk images on selected drives (consulting the database for information on the image to apply for a particular "computer" as well as retrieving that image). So, the "wipe disk" step can be combined with the "install system" step to cut the time required to "build" a system (i.e., the system -- OS + apps -- is installed in the time it takes to wipe the disk!)

[The database then logs the disk and/or computer into "Inventory" so you can see what stuff you have on hand]

The goal of all this is to make it easy for an untrained user (volunteer) to wipe disks or build computers reliably and in very little time (if a volunteer only is present for 3 hours, you want to be able to get results from that time, not just "partial builds" -- which would be another opportunity for errors to creep into the process). This would make it easier to feed an eBay person with tested/completed systems for sale (to subsidize the "charity" aspect of the organization).
Curious.George is offline   Reply With Quote
Old 04-12-2019, 03:55 PM   #7
brethin
Badcaps Veteran
 
Join Date: Dec 2008
City & State: Owensboro, KY.
My Country: USA
Line Voltage: 120VAC 60Hz
I'm a: Professional Tech
Posts: 1,708
Default Re: Disk conidence test while wiping

so in that long ass post you still failed to answer his question, typical of your posts, you ramble on and on.☹️
brethin is offline   Reply With Quote
Old 04-12-2019, 04:10 PM   #8
Curious.George
Badcaps Veteran
 
Join Date: Nov 2011
Posts: 1,304
Default Re: Disk conidence test while wiping

Quote:
Originally Posted by brethin View Post
so in that long ass post you still failed to answer his question, typical of your posts, you ramble on and on.☹️
Dear illiterate moron,

Here is his question:

"...will the company or charity sell on ebay"

Now, if you got your head out of your ass and wiped the fecal matter out of your eyes, you'll note:

"The goal of all this is to make it easy for an untrained user (volunteer) to wipe disks or build computers reliably and in very little time (if a volunteer only is present for 3 hours, you want to be able to get results from that time, not just "partial builds" -- which would be another opportunity for errors to creep into the process). This would make it easier to feed an eBay person with tested/completed systems for sale (to subsidize the "charity" aspect of the organization)."

If my posts bother your tiny mind, why don't you add me to your IGNORE LIST. Perhaps one of your 8-year old neighbors could show you how to do that, eh?

Moron.

<plonk!>
Curious.George is offline   Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump



Badcaps.net Technical Forums 2003 - 2019
Powered by vBulletin ®
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
All times are GMT -6. The time now is 04:29 AM.
Did you find this forum helpful?