Hacker News new | ask | show | jobs
by DanielBMarkham 6354 days ago
Wow. Simply -- wow.

I was amazed at really seeing the HD bottleneck minimized this way. I always knew about it, but -- wow.

3 comments

Would be interesting to see a baseline against a RAIDx24 with conventional HDDs.
Agreed. I wouldn't go so far as to call it meaningless without something to compare it to, but I'd imagine conventional disks would still be very fast in that setup.
Yeah - and perhaps for certain scenarios it might be faster? (Pure conjecture on my part).

So - if seek time wasn't a big factor, a fast conventional drive could be faster? And if you have a very high level of striping (ie. 24 drives), seek time might be significantly mitigated.

Given the tested max throughput of the Hitachi 1TB SATAs (http://www.storagereview.com/Testbed4Compare.sr) you should be able to get ~2GBps with 2/3 the drives (16 instead of 24) and near triple the storage space (16TB instead of 6TB). They're cheaper, too, of course. That's just theory, though. It would probably be difficult to get all drives "bursting" at the same time but anyway, this kind of looks like a "jumping into jeans" viral published by Samsung.
I think this bottleneck is easily (and still amazingly) noticed when storing things in RAM rather than on disk. For instance, immediately after a restart of our production database it can take up to a minute to run certain queries. Give it a few seconds to load everything into RAM, though, and it's nearly instant. I've noticed the same thing when switching from file-based to RAM-based queuing.

Everybody thinks, "duh, RAM (or SSD or w/e) is faster than disk" but sometimes these sorts of real-world tests still amaze.

Now, as a software guy, can I stop hearing about how software "bloat" keeps absorbing all the hardware advances? If hard drives had been advancing, errr, at all, this would be considered normal performance.

And I daresay that in three or four years, this will indeed be considered "normal performance", if SSDs continue to develop.

All hail the SSDs! Maybe in a couple of years we actually will be able to feed our octo-cores some data to work on!

> Now, as a software guy, can I stop hearing about how software "bloat" keeps absorbing all the hardware advances?

Why should you stop hearing it? Hardware is getting faster and SW is using the capacity.

> If hard drives had been advancing, errr, at all,

They have been, just not uniformly. There have been huge increases in capacity. Less so in transfer rate, but still significant. Seek time hasn't been improving much and rotational latency is stuck.

In the same time, arithmetic and logical operations have essentially become free. Branch mispredictions are a big deal. Cache misses are a killer but memory bandwidth is about to become the biggest problem. (It already is for some people.)

When someone wants to talk about bloat, the odds of Microsoft Office being trotted out are around 80%.

Yet, here we see that with a modern system that is mostly only special in the I/O department that Microsoft Office as a whole opens in .5 seconds, which, believe me, is radically faster than Office 95 opened back in the day it was still cutting edge.

Yes, it was a powerful system, but the only thing that was radically more powerful that what anybody can buy for cheap was the I/O system.

Maybe it's "bloat" in the abstract if programs are simply "larger" than they used to be, but really, who cares about bloat if hardware is actually outpacing software development... in everything but the I/O hardware.

I'm just a bit tired of people babbling on about software bloat when this pretty conclusively proves its the I/O subsystem that is the problem.

> I'm just a bit tired of people babbling on about software bloat when this pretty conclusively proves its the I/O subsystem that is the problem.

This proves nothing of the sort. (Office95 was bloated too....)

The time to boot is large because software is huge, scattered all over the disk, and has unnecessary dependencies.

It may be convenient to believe that everything fits in L1 cache, but doing so often results in slow programs. The same principle applies here. Some operations cost more than others. Software which uses slow operations will run slower than software that doesn't.

To put it another way, big O is the start, not the end.

I wonder if there is a market for open desktop software that is designed to be as fast as possible and still get the job done.

Imagine a WYSIWYG editor that is actually 2^10 times faster than those made 15 years ago.

I think uTorrent has proven small can be loved.