Hacker News new | ask | show | jobs
by haney 1379 days ago
It’s kind of coming from the opposite direction, but it feels like Postgres extensions solve many of the needs you’re describing. They allow a developer to use the underlying database engine while extending it to include domain specific primitives.
1 comments
jandrewrogers 1379 days ago
That's the thing, I've written a lot of Postgres extensions over the decades. Those extensions are fundamentally limited by the underlying architecture of Postgres in myriad ways, though I've been very creative about pushing the limits. In the big picture the kinds of extensions that work well in Postgres are for a relatively narrow set of domains that roughly match what Postgres was designed for but for which it may lack some specific features.

The limitations and capabilities of a database are largely architectural. It isn't something you can "extend", it is either there or it isn't. This is why being able to codegen a purpose-built architecture would be interesting; instead of trying to force a domain into an architecture that doesn't fit well, we could produce an architecture that is a perfect fit for the domain while inheriting many of the basic quality of life features you expect.

link
HereBeBeasties 1379 days ago
I'd be fascinated to know what sort of things it is hard to write Postgres extensions for and how that interplays with their existing architecture choices, if you have the time to give some examples.
link
_vvhw 1379 days ago
You know, we almost wrote TB as a Postgres extension!

I think it would have been the right thing to do 5 years ago, before some of the groundbreaking research that came out in 2018 like fsyncgate and “Protocol-Aware Recovery for Consensus-Based Storage” that really changed the way that distributed databases need to be designed [1].

These days we also have io_uring, Deterministic Simulation Testing and safer systems languages (Rust, Zig). And high availability, i.e. consensus, almost has to be part of the (distributed) database going forwards.

Beyond this, in the case of TB, a Postgres extension didn't satisfy our storage fault model, or our design goals of gray failure tail latency tolerance and static memory allocation.

What we're excited about with TB, is also this vision that people will one day be writing their own extensions for TB, swapping out the accounting state machine for another, with TB doing all the distributed heavy lifting.

[1] “Let's Remix Distributed Database Design!” https://www.youtube.com/watch?v=rNmZZLant9o

link