Announcement

Collapse
No announcement yet.

Partitioning based on media transfer rate zones

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    Partitioning based on media transfer rate zones

    You may know this, but for over 25 years hard disk drive manufacturers made use of zoned recording (more sectors on the outer tracks and fewer on the inner tracks) to maximize media recording density, but the inner tracks are slower as a result.
    For a piece of custom test equipment I am building (which would stream long patterns stored on the drive), a 500GB 2.5" drive was split into four partitions: the first 40% for 70-80 MB/sec, 20% for 60-70 MB/sec, 20% for 50-60 MB/sec and the rest for 40-50 MB/sec.

    Have you considered such partitioning?
    My first choice in quality Japanese electrolytics is Nippon Chemi-Con, which has been in business since 1931... the quality of electronics is dependent on the quality of the electrolytics.

    #2
    Re: Partitioning based on media transfer rate zones

    wont work,
    modern drives re-map the data virtually.

    if your after speed, get an "A/V" drive.
    the firmware is written to keep the data on consecutive sectors to reduce seeks.

    the downside to that aproach is you get heavily used areas on the platters that wear faster - hence the regular firmware scattering the data all over the platters.

    Comment


      #3
      Re: Partitioning based on media transfer rate zones

      I've setup a few short stroking hard rives back in the day but they were only two partitions.
      Since SSDs are cheap and far faster, why bother ?

      Comment


        #4
        Re: Partitioning based on media transfer rate zones

        Yes this works but if you plan on using the whole disk equally, you're killing yourself back to square one. "Short stroking" is the name of the game but you waste a lot of disk space.

        I did this decades ago with swap partitions to increase swap throughput. However if I needed to access the rest of the disk constantly, it would be a net wash or potential loss compared to sticking swap in the middle of the used area.

        Comment


          #5
          Re: Partitioning based on media transfer rate zones

          That's what I do for RAID-0. (At least for the OS and games)

          Need a bigger partition? Then just create another RAID array.
          ASRock B550 PG Velocita

          Ryzen 9 "Vermeer" 5900X

          16 GB AData XPG Spectrix D41

          Sapphire Nitro+ Radeon RX 6750 XT

          eVGA Supernova G3 750W

          Western Digital Black SN850 1TB NVMe SSD

          Alienware AW3423DWF OLED




          "¡Me encanta "Me Encanta o Enlistarlo con Hilary Farr!" -Mí mismo

          "There's nothing more unattractive than a chick smoking a cigarette" -Topcat

          "Today's lesson in pissivity comes in the form of a ziplock baggie full of GPU extension brackets & hardware that for the last ~3 years have been on my bench, always in my way, getting moved around constantly....and yesterday I found myself in need of them....and the bastards are now nowhere to be found! Motherfracker!!" -Topcat

          "did I see a chair fly? I think I did! Time for popcorn!" -ratdude747

          Comment


            #6
            Re: Partitioning based on media transfer rate zones

            Originally posted by diif View Post
            I've setup a few short stroking hard rives back in the day but they were only two partitions.
            Since SSDs are cheap and far faster, why bother ?
            For the amount of data on my NAS, SSD still isn't feasible.... to small and too expensive.
            <--- Badcaps.net Founder

            Badcaps.net Services:

            Motherboard Repair Services

            ----------------------------------------------
            Badcaps.net Forum Members Folding Team
            http://folding.stanford.edu/
            Team : 49813
            Join in!!
            Team Stats

            Comment


              #7
              Re: Partitioning based on media transfer rate zones

              Originally posted by Topcat View Post
              For the amount of data on my NAS, SSD still isn't feasible.... to small and too expensive.
              I got a Western Digital Black 1 TB spinner, just in case my Samsung TLC SSDs take a dump.
              ASRock B550 PG Velocita

              Ryzen 9 "Vermeer" 5900X

              16 GB AData XPG Spectrix D41

              Sapphire Nitro+ Radeon RX 6750 XT

              eVGA Supernova G3 750W

              Western Digital Black SN850 1TB NVMe SSD

              Alienware AW3423DWF OLED




              "¡Me encanta "Me Encanta o Enlistarlo con Hilary Farr!" -Mí mismo

              "There's nothing more unattractive than a chick smoking a cigarette" -Topcat

              "Today's lesson in pissivity comes in the form of a ziplock baggie full of GPU extension brackets & hardware that for the last ~3 years have been on my bench, always in my way, getting moved around constantly....and yesterday I found myself in need of them....and the bastards are now nowhere to be found! Motherfracker!!" -Topcat

              "did I see a chair fly? I think I did! Time for popcorn!" -ratdude747

              Comment


                #8
                Re: Partitioning based on media transfer rate zones

                Originally posted by Topcat View Post
                For the amount of data on my NAS, SSD still isn't feasible.... to small and too expensive.
                And, likely, offers more performance than you'd actually *need*!

                At least in my case, my NAS is accessed much like my safe deposit box; occasionally fetch something and bring it home for normal use.

                So, my NAS is "cold", most days/weeks. And, I have to pick which physical volumes to install in it prior to actually accessing any of that data (list of every volume's contents can be browsed in an on-line database).

                Comment


                  #9
                  Re: Partitioning based on media transfer rate zones

                  I run 3 NAS units, 2 are very active, 1 with movies 32 TB raid 6, 1 with music 8 TB raid 6 and 1 for backups 48TB raid 6.

                  Comment


                    #10
                    Re: Partitioning based on media transfer rate zones

                    Originally posted by brethin View Post
                    I run 3 NAS units, 2 are very active, 1 with movies 32 TB raid 6, 1 with music 8 TB raid 6 and 1 for backups 48TB raid 6.
                    So is mine. This is my NAS setup. Each system pulls 280w off the grid. The second system in this NAS is a backup/mirror of the first one, and is off 99% of the time.....but I have considered ways to make it more efficient, such as ITX PC-on-a-chip setups, they don't require much raw horsepower. I run WD greens and WD reds, all 5400RPM spinners. THey run cool, efficient, and plenty fast enough for multiple HTPC's to stream from simultaneously....and well, of course store files. Each system has 2x 8TB RAID-10 arrays. They're close to capacity, but not crazy full....raw DVD ISO's are ~8gb each....then tunes....then work-related files.
                    <--- Badcaps.net Founder

                    Badcaps.net Services:

                    Motherboard Repair Services

                    ----------------------------------------------
                    Badcaps.net Forum Members Folding Team
                    http://folding.stanford.edu/
                    Team : 49813
                    Join in!!
                    Team Stats

                    Comment


                      #11
                      Re: Partitioning based on media transfer rate zones

                      Originally posted by Topcat View Post
                      So is mine. This is my NAS setup. Each system pulls 280w off the grid. The second system in this NAS is a backup/mirror of the first one, and is off 99% of the time.....but I have considered ways to make it more efficient, such as ITX PC-on-a-chip setups, they don't require much raw horsepower. I run WD greens and WD reds, all 5400RPM spinners. THey run cool, efficient, and plenty fast enough for multiple HTPC's to stream from simultaneously....and well, of course store files. Each system has 2x 8TB RAID-10 arrays. They're close to capacity, but not crazy full....raw DVD ISO's are ~8gb each....then tunes....then work-related files.
                      I'm running a 14 drive 1.75gb RAID 10 with a 4tb weekly backup drive. Not terribly efficient but it fits in my rack, so I'm happy.
                      sigpic

                      (Insert witty quote here)

                      Comment


                        #12
                        Re: Partitioning based on media transfer rate zones

                        Originally posted by ratdude747 View Post
                        I'm running a 14 drive 1.75gb RAID 10 with a 4tb weekly backup drive. Not terribly efficient but it fits in my rack, so I'm happy.
                        My NAS is "distributed" (many hosts on an internet) and operates at a much higher level of abstraction than a NAS "appliance". I.e., there are media that hold data, a VTOC that keeps track of "what's where" (and other metadata), interfaces that provide access (hardware and software) to the media (and, thus, the data) and rules that govern how data is replicated and "vouched for". Presently, about 100 spindles that can be powered up in various combinations on a score of hosts (so, HUGE processing and transport bandwidth).

                        Unlike an appliance, I place no constraints on the "availability" of data. The media holding it may be offline -- that doesn't make it corrupt, rather, just "having a longer access time"! Whenever a volume "appears", scanning (reverification) of its contents can resume -- as it is being otherwise accessed. If a particular volume is needed, the user is prompted to "mount it" (e.g., load a CD into a drive, somewhere that can be found).

                        I also don't care what type of media is used -- spinning rust, solid state (not just SSDs), tape (until recently), WORM (weening myself of this, presently), optical, etc. As long as there is an interface that can make that data available to the "system", it's OK. Interfaces can be IDE, SATA, SAS, SCSI, SCA, FC-AL, iSCSI, USB, 1394, CF, SD, SSD, etc. -- why should the NAS care?! It's just "storage"! (why do all of the physical volumes in the NAS have to share a common interface?)

                        I use an RDBMS as the VTOC -- as well as for tracking housekeeping data for the Array (i.e., when was the last time this datum was VERIFIED?).

                        And, the rules are just software -- algorithms.

                        These looser constraints give me more flexibility in what I can store, how I store it and how much effort I want to have expended (on my behalf) to preserve any particular datum ("file"). I can choose to store five different copies of an object ("file") in five different places ("containers") on five different media ("volumes") of five different physical characteristics ("media types") behind five different interfaces hosted by five different OS's. Yet, to *me*, the user, they are all functionally interchangeable! Give me ANY of those copies and I've got the data that I want/need!

                        [I.e., the CD/DVD originals from which I created each of the ISOs can be genuine parts of the NAS store -- even though they are read-only, removable, slow, small, etc!] If an ISO gets corrupted, the RDBMS tells me where to find another copy of the files IN the ISO that have been affected -- so the ISO (which is a container!) can be recreated. Ditto a RAR/ZIP archive, etc.

                        I can implement any variety of RAID (no striping) on any collection of files/objects just by building those desired relationships in the RDBMS that tracks all of the contents of the various volumes. And, at a granularity appropriate for the data involved. Volumes need not be the same size or have similar access characteristics (e.g., one half of a RAID 1 can be a CD while the other half is made of rust)

                        Because the RDBMS is available 24/7/365, I can browse the NAS's contents without having any of those media actually on-line. And, I can define the schema to let me annotate entries in any way that makes sense, to me (beyond just a "filename").

                        I can expand the array one volume at a time without having to "rebuild" anything. E.g., the 12 2T SAS drives I picked up this week magically increase the array's capacity without any special overhead -- just start copying files onto them like any other media!

                        Finally, I can deal with subsets of the array outside of its scope -- just move the drive to another machine and access it as a "normal" drive (e.g., a USB disk can be carried to a friend's home and used there; then reattached to the array as if it was never gone!). Because of this, I can recover/repair/access multiple volumes concurrently WITHOUT requiring special appliance hardware! (what happens when the power supply in your appliance shits the bed? or, the NIC fails?)

                        Comment


                          #13
                          Re: Partitioning based on media transfer rate zones

                          Originally posted by Topcat View Post
                          Each system pulls 280w off the grid. The second system in this NAS is a backup/mirror of the first one, and is off 99% of the time.....but I have considered ways to make it more efficient, such as ITX PC-on-a-chip setups, they don't require much raw horsepower. I run WD greens and WD reds, all 5400RPM spinners. THey run cool, efficient, and plenty fast enough for multiple HTPC's to stream from simultaneously....and well, of course store files.
                          I've had bad luck with Greens (I don't even bother rescuing them, anymore).

                          What sort of sustained throughput can you get from them (or the reds)? I'm building a 50T datastore for the RDBMS in my "disk sanitizer" to store compressed disk images from which to mass-produce machines. I'm stuck trying to balance power consumption (it would be nice to only have the drives that were serving up images AT THE TIME actually spinning) vs throughput -- I want to be able to build multiple machines concurrently. I either have to actively manage power on the individual drives (i.e., spin down those that aren't needed at the moment) or opt for drives that will do this on their own (though they are usually lower performing).

                          Comment

                          Working...
                          X