[ncruces]

I don't know much about DuckDB's architecture.

Wasm is fine for compute (though concurrency is still a somewhat open question).

To have Wasm talk to the outside world, you need “host calls” where the guest calls the host.

On a browser that's Wasm calling JavaScript. On my Go driver, it's Wasm calling Go.

For server side, there's also a standard set of “host calls” modeled around POSIX/Linux syscalls called WASI.

I could've build my project around WASI, but WASI is rather limited (and SQLite support for WASI was more limited even, it's improved a bit since). DuckDB might work out-of-the-box this way.

I, instead, took advantage of SQLite's architecture and replaced its VFS layer with one in Go: https://sqlite.org/vfs.html

So SQLite in Wasm is just doing compute, and I do all the OS level stuff in Go. No need for Wasm concurrency, cause I can load multiple instances of my Wasm which act like independent OS processes that communicate through the filesystem (SQLite excels at this).

As I said, I dunno how well all those decisions would map to DuckDB.

[koeng]

> So SQLite in Wasm is just doing compute, and I do all the OS level stuff in Go. No need for Wasm concurrency, cause I can load multiple instances of my Wasm which act like independent OS processes that communicate through the filesystem (SQLite excels at this).

Interesting. So when I am running concurrent readers using your package, it is just loading multiple instances of the wasm code? (I bottleneck to a single writer in the application)

[ncruces]

Yes.

Each connection lives in its own isolated sandbox, and only communicates with other connections through the “file system” (which is a virtual abstraction, actually).

WAL mode is the “exception”: a few pages of the sandbox's memory are mapped to a file, and shared by all connections to the same database.

Each sandbox is single threaded, and mostly lock free, does all its business in the calling goroutine, and regularly checks back with the Go runtime to play nice with the Go scheduler.

It's a bit like an OS running multiple processes, with the VFS layer handling all syscalls.

[TechDebtDevin]

This is why I've switched to go and am never looking back.