Caching HDD with SSD

[eccerr0r]

Well for those people who don't have 4TB SSDs, wonder if anyone has gone the cache route?

Well, what I'd like to do - I have a 4TB disk array (RAID5) and would like to add an SSD cache to speed up operations.

Don't worry about the implementation - though I am using Linux LVM and planning on using LVM's built-in disk cache functionality, but again don't worry about it. Assume that I can suddenly add an SSD cache.

What I wonder is how much of an improvement I'll see in multiple I/O consumers. The disk array is currently being accessed by multiple IO streams and is getting killed by this fact, as expected. I wonder how much of a SSD would cause an improvement in interactive performance or will it be mandatory that I have an SSD of the same size of my array to get any benefit? I'm worried about write performance as well, unfortunately this will be bottlenecked by the underlying HDD, perhaps I am forced to use writeback caching to improve performance?

I'm hoping I can get away with a much smaller SSD than the 4T on the array. Also wonder how badly will the SSD get trashed in terms of write cycles/lifetime?

Anyone using such a setup?

[stj]

Re: Caching HDD with SSD

the ssd will probably die.
the profesional method - that is now cheap on ebay,
is to use a buffered raid card.
these cards have a big ram/sim/dim on them backed by lithium batteries. (incase of a crash/power loss)

[eccerr0r]

Re: Caching HDD with SSD

I'd imagine the SSD would last as long as an SSD in the same situation, depends on how much you write to it, though RAID may amplify writes a bit. However a 64G SSD is cheaper than 64GB DDR4 on a RAID card... not as fast but still much faster than mechanical HDDs...

[stj]

Re: Caching HDD with SSD

you dont need 64gig ram on the raid card, 4-16 is more than enough.
the card only uses it as a buffer between the system and the spinning harddrives.

[eccerr0r]

Re: Caching HDD with SSD

I haven't been able to find any cheap ones that have more than 512MB RAM, and based on my datasets I think I do need at least about 4GB or so as that's the most commonly used datasets that should be cached... seen any? 16GB would be nice, and then 64GB SSDs are cheap now?

[eccerr0r]

Re: Caching HDD with SSD

Well I implemented it when I did a server cleanup. Used a 180G SSD, there's quite a bit of stuff that sped up...

... and then everything went down hill. Yow.

Turns out, after cleaning out a bad fan that was seized, the fan now vibrates horribly and causes one of the hard drives to error out making the cache system useless.

%$#*(%Ing 40mm fans. I stuck a wire into it to shut up the stupid fan and disk speeds went back up to normal.

[stj]

Re: Caching HDD with SSD

what uses a 40mm fan? i'v not seen one in years.
i used to replace them with 50mm - the smallest fan that actually moves more air than it heats!!

[diif]

Re: Caching HDD with SSD

what uses a 40mm fan? i'v not seen one in years.
i used to replace them with 50mm - the smallest fan that actually moves more air than it heats!!

Anything 1U, network switches mostly.

[brethin]

Re: Caching HDD with SSD

Ssd makes a bad cache device, use a ram card.

[stj]

Re: Caching HDD with SSD

Anything 1U, network switches mostly.

you could use a 50mm on a slanted plate to lower it's height.
or just use a centrifugal type like on laptops

[diif]

Re: Caching HDD with SSD

you could use a 50mm on a slanted plate to lower it's height.
or just use a centrifugal type like on laptops

No need, I have lifetime supply of Sunon Maglev 40mm fans.

[eccerr0r]

Re: Caching HDD with SSD

Cache system actually works fairly well now, when the underlying HDD is actually working properly. Cold misses suck as usual but once it warms up, ahh...

Granted that the cache:size ratio is fairly large since I targeted only the hot spots, better than 1:8 ratio for them. As long as I don't (frequently) touch the uncached portions...

[Uranium-235]

Re: Caching HDD with SSD

At one time they made hard drives with a SSD cache. They were called SSHD. They were total shit

[eccerr0r]

Re: Caching HDD with SSD

Buy underspecced, expect a miracle, be disappointed...

[eccerr0r]

Re: Caching HDD with SSD

I have to say, the cache system is working very well. I allocated about 128GB to cache 800GB right now (rest of the 4T I don't cache), very nice, it covers up a lot if not almost all of the disk slowness, even during RAID scrub night. My disk array is horridly slow during biweekly maintenance night, it's still slow but the cache hides the latency -- I can actually do something with the computer during scrubbing now!

...Which is kind of moot anyway, I explicitly set scrub night to be when I'm most likely not using the computer so the slowness is hidden away already...