Hacker News new | ask | show | jobs
by sreekanth850 176 days ago
Unpopular Opinion: Database were designed for 1980-90 mechanics, the only thing that never innovates is DB. It still use BTree/LSM tree that were optimized for spinning disc. Inefficiency is masked by hardware innovation and speed (Moores Law).
2 comments
cmrdporcupine 176 days ago
There's plenty of innovation in DB storage tech, but the hardware interface itself is still page-based.

It turns out that btrees are still efficient for this work. At least until the hardware vendors deign to give us an interface to SSD that looks more like RAM.

Reading over https://www.cs.cit.tum.de/dis/research/leanstore/ and associated papers and follow up work is recommended.

In the meantime with RAM prices sky rocketing, work and research in buffer & page management for greater-than-main-memory-sized DBs is set to be Hot Stuff again.

I like working in this area.

link
sreekanth850 176 days ago
Btrees are not optimal for SSD, and the only reason we still use them is legacy constraints of page-oriented storage and POSIX block interfaces.We pay a lot of unnecessary write amplification, metadata churn, and small random writes because we’re still force-fitting tree structures into a block device abstraction.
link
cmrdporcupine 176 days ago
I don't think we're disagreeing. But the issue is at the boundary between software and hardware, which the hardware device manufacturers have dictated, not further up.
link
nly 176 days ago
Optimising hardware to run existing software is how you sell your hardware.

The amount of performance you can extract from a modern CPU if you really start optimising cache access patterns is astounding

High performance networking is another area like this. High performance NICs still go to great lengths to provide a BSD socket experience to devs. You can still get 80-90% of the performance advantages of kernel bypass without abandoning that model.

link
gethly 176 days ago
> The amount of performance you can extract from a modern CPU if you really start optimising cache access patterns is astounding

I think this was one, and I want to emphasise this, of the main points behind Odin programming language.

link