[masklinn]

Seems like a potentially interesting project to get rid of sqlite's compatibility baggage e.g. non-strict tables, opt-in foreign keys, the oddities around rowid tables, etc... as well as progress the dialect a bit (types and domains for instance).

[wis]

But the article mentions that they intend to have full compatibility:

  > Our goal is to build a reimplementation of SQLite from scratch, fully compatible at the language and file format level, with the same or higher reliability SQLite is known for, but with full memory safety and on a new, modern architecture.

[glommer]

author here: fully compatible at the language and file format level.

Further down the post I actually call out explicitly that we do intend to get rid of some of the baggage.

[ncruces]

If you "intend to get rid of some of the baggage" you won't be fully compatible.

libSQL already isn't fully compatible: as soon as you add a RANDOM ROWID table, you get "malformed database schema" when using the (e.g.) sqlite3 shell to open your file (also Litestream doesn't work, etc).

And that's fine, as there probably is no better way of doing what you needed to do. But it's also taking what SQLite offers and breaking the ecosystem, under the covers of "we're compatible" without ever calling out what compromises are being made.

Note how the SQLite documentation that introduces STRICT tables very clearly documents the backwards compatibility issues of the feature and how to get around them: https://sqlite.org/stricttables.html#backwards_compatibility

You also never got round to documenting the internal Virtual WAL APIs you exposed. This is something where SQLite is lacking, where you could've made an impact without any compatibility issues, and pressure upstream to release something by doing it first/better. Alas, you did it for Turso's exclusive benefit.

[rnmkr]

That's fine though. Full compatibility doesn't have to mean full backwards compatibility. I think of it as what's Typescript to Javascript.

[ncruces]

Once you compile your Typescript to Javascript, Javascript runtimes can run it, Javascript code can call it, etc. Even source maps work.

Once you start using libSQL features, SQLite tools will simply stop working with your databases.

That means the sqlite3 shell stops working, backup solutions like Litestream and sqlite-rsync stop working, SQLite GUIs like SQLiteStudio stop working, forensic and data recovery tools start giving will have a harder time working, etc.

Maybe it's all worth it, but it's not full compatibility, and it should at least be documented.

[throwawaymaths]

i would guess "full memory safety" is going to be impossible, at least at compile time. I'd guess that if for no other reason than performance SQLite uses data oriented techniques that effectively reduces pointers to indices, which will no longer have ownership or lifetime tracking in the rust compiler.