[clarkema]

That's an interesting point that I should perhaps have covered in the original article.

The real code that this is based on is in fact hidden behind a GenServer for this exact reason -- to maintain the expectations of other Elixir code that has to interact with it. The advantage of the escape hatch, as another commenter mentions, is allowing efficient sparse mutations of a large chunk of data, without having to pay a copy penalty every time. I definitely wouldn't recommend sharing the db handle widely.

[rkangel]

Did you consider a port (written in Rust) instead of a NIF?

When you're presenting a GenServer like message passing interface a port is a natural fit, with none of the risks related to linking a NIF into the VM itself.

(admittedly those risks are much lower with Rust than C)

[clarkema]

In our case one of these NIF stores is created per user for a specific task; ironically, with the amount of polish that Rustler puts around NIFs I suspect it would have been more work and more risk to go down the port route and manage everything manually.

[lvass]

Have you measured performance? If mutating from Elixir like this can bring serious benefits, maybe there's a place for mutable versions of libraries like Explorer and Nx.

[clarkema]

Explorer does actually use Rust (and polars) for a lot of its work -- its one on the libraries I looked at while figuring out my memory management issues.

[lvass]

But would it benefit from mutating the value of one reference? At the moment it does not do that, right?

[clarkema]

No, it doesn't -- looking at the website that's an explicit trade-off of pure performance vs 'Elixir-ish-ness'. It would certainly break a lot of expectations to have data mutating like that without it being hidden away somewhere, so I can understand why they went that way.

In my case the data I'm dealing with is more of a store than a single data item, so I'm leaning on the example of things like ETS. Also it's within a single application rather than being a large generally-available library, so the trade-offs are different. It would be interesting to know if they did tests though.

[lawik]

Probably not ideal since Polars builds on Apache Arrow and that tends to want to treat the structure as immutable if I recall correctly.