[laserbeam]

Keep in mind, the thing that matters most for tigerbeetle’s speed is they are domain specific. They know exactly upfront how data looks like. They don’t have to be general purpose.

The style of programming does work for general purpose computing, but their requirements enable a significant % of “orders of magnitude faster than postgress”.

[vendiddy]

While your take makes sense, I feel like even general purpose programming frameworks should be orders of magnitude faster for what they are doing.

[laserbeam]

Definitely! It's just your benefits aren't going to be "all" the orders of magnitude speedups, just some of them. And I agree, that's nothing to shy away from.

[josephg]

Yeah; and we have plenty of other general purpose databases which can run laps around postgres and sqlite.

There's plenty of optimisations postgres and friends leave on the table. Like, the postgres client driver could precompile queries and do query planning on behalf of the database. It would introduce some complexity, because the database would need to send information about the schema and layout at startup (and any time it changed). But that would remove a lot of unnecessary, repeated makework from the database process itself. Application servers are much easier to scale than database servers. And application servers could much more easily cache the query plan.

Weirdly, I think part of the problem is economic. I think there's a lot of very smart people around with the skills to dramatically improve the performance of databases like postgresql. Across the whole economy, it would be way cheaper to pay them to optimize the code than needing companies everywhere to rent and run additional database servers. But it might not be worth it for any individual company to fund the work themselves. Without coordination (ie, opensource funding), there's not enough money to hire performance engineers, and the work just doesn't happen.

[gregw2]

Would odbc/jdbc protocols have to change to support client-planned queries?