Hacker News new | ask | show | jobs
by samsk 962 days ago
Hmm, when choosing database solution for a new project I'm not selecting sql dialect (like postgresql sql), but stability and ecosystem. So having this as an extension to postgresql, and possibility to combine it with other extensions it would be great, but reimplementation is a no go here.

And I can not use it for existing projects, because again extensions, and I surely don't want to findout how your implementation differs from the mainstream postgres...

1 comments

Knowing that extensions are very important to you is great feedback for us. As we hear more about what users' requirements are, it helps us better plan for the future.

Regarding any differences from mainstream Postgres, you can look to how we've handled Dolt, which is production-ready. It targets MySQL, just as DoltgreSQL targets PostgreSQL, and it recently achieved 99.99% correctness according to a set of roughly 6 million tests (https://www.dolthub.com/blog/2023-10-11-four-9s-correctness/). This test is not a definitive stance that we are exactly 99.99% the same as MySQL, but it's a good general guide to how we approach our compatibility, and how serious we are in that regard.

Sadly though, the full versioning capabilities would not work as an extension to Postgres. We looked into it before we settled on our current approach. I talk a bit about it in the blog post as well. To truly allow versioning in the same capacity that Git does for source code, it required us to either fork Postgres and spend years reimplementing all of the work that we've done in Dolt just to get to where we are today, or choose the path that gets something out quickly, and allows us to have the very conversation that we're having right now.

That was actually the very reason for deciding to host an announcement that we're working on it. Many people have said that they'd like Dolt but for Postgres, however they've not said whether they need the Postgres binary specifically, the ecosystem, the syntax/wire protocol, etc. This announcement gives us the opportunity to receive that feedback.

Where are the 0.01% differences? When I'm trying to commit transactions? During select? Just not supporting some esoteric stored function syntax?

And most importantly, how does Dolt compare under heavy load, at the limits of server memory or bandwidth or CPU or disk thoroughput? You can assume an SSD and a multicore processor for purposes of answering.

Thank you very much. You are competing in a field where trust is extremely difficult to acquire - and the consequences to a lead dev for choosing Dolt[greSQL] could end his career. Nobody ever got fired for choosing the incumbent, as variations on the saying go.

The sqllogictests are better used as a judgement of how accurate our results are to MySQL's results given a very large range of statements (around 6 million tests). They are not, however, indicative of the entire feature set. That's to say, there are a few things that are not yet supported in Dolt, however they are the more esoteric features. Things like transactions, etc. are what I would consider "core" SQL, and we are very keen on making sure those work exactly as you'd expect

There are two key points regarding performance that I'd like to mention, and that's that Dolt uses higher-than-expected disk, and we also use more memory than MySQL by comparison. The disk usage is due to our optimization of speed at the sacrifice of disk accretion via temporary storage, which we've decided is a fair trade off considering disk is very cheap (and we're working on making this tunable so users can decide on speed vs disk efficiency). Memory usage is a bit more complex, and I'm not the employee to comment too much about it, but both of these issues are being worked on to reduce their impact. With that in mind, our performance is comparable to MySQL as long as the machine limits are not being reached, however I'd expect us to be a bit slower once those limits are reached, simply due to the extra complexity that we're managing.

Lastly, trust is something that can only be built over time. In 10 years, I'm sure that there will be no doubt of our stability, and at that time I can see Dolt and DoltgreSQL becoming the de-facto databases used for relational storage. Of course, I may be a bit biased :)

Trust is a