[narnarpapadaddy]

I agree that it’s not inherent to emitting machine code but I do think it reflects a different set of priorities.

In extremely high performance code you use different data structures and algorithms and change your approach to memory allocation. TigerBeetle famously does all memory allocation once on startup.

Roc is attempting to make a similar set of trade-offs in their compiler as Zig, so it makes sense that the author finds many shared patterns.

[steveklabnik]

I do think it reflects different priorities, but one of those differences is that from my perspective, safety and performance are not inherently at odds. Yes, sometimes it is needed, but not as much as some people seem to think. Sometimes, it also means writing code in ways that communicate things to the compiler that you may not think of if you're not used to thinking in this manner.

A lot of the ways in which the zig compiler works doesn't use pointers, it uses indices. This stuff is easier to write as safe code, not less easy.

> Roc is attempting to make a similar set of trade-offs in their compiler as Zig, so it makes sense that the author finds many shared patterns.

I do think that that makes sense, but it also doesn't mean that they have to. I am doing a compiler project that takes a lot of inspiration from Zig (as my language currently inherits some major things from Zig, and I also care a lot about compiler performance) and it's written in Rust, and does not use much unsafe code (outside of the usual suspects of FFI in the runtime, etc).

[torginus]

> A lot of the ways in which the zig compiler works doesn't use pointers, it uses indices. This stuff is easier to write as safe code, not less easy.

I respectfully disagree. This is only true if you view malloc as qualitatively different from a piece of code that gives you an index for a free object in an object pool.

Provided you don't ask/give back memory from/to the OS, what malloc is doing is giving you an index (pointer) into a pool of bytes, while manipulating an internal bookkeeping structure.

Use after free is just you using an index after said bookeeping structure has marked that piece of memory as available for something else (and perhaps claimed already).

If you have an array of Node structs to represent a graph (like the AST in Zig), and use indices to represent references, you have essentially zero protection in Rust that helps you with finding Node-s that have been used for something else etc.

The 'asking memory from the OS' aspect for malloc doesn't really change the safety of your language compared to this where it matters - if you do use-after-free on a page claimed by the OS, you get a segfault, which immediately tells you there's a problem, which is much better than silent corruption.

At least with malloc, you get debug allocators, or other features that can help you in this case. If you are careless with indices in an object pool, and overwrite stuff, essentially, it's up to you to figure out what went wrong and you have no tools to help you.

[steveklabnik]

I fully agree that there are similarities here. But we disagree about some of the details.

> you have essentially zero protection in Rust that helps you with finding Node-s that have been used for something else etc.

The most obvious technique is generations. You can of course do that in Zig as well.

> if you do use-after-free on a page claimed by the OS,

This assumes that you're working in the context where there is an OS. That isn't always the case. Also, there are other cases than just use-after-free: for example, compilers will optimize around null pointers being UB, which can cause other problems, whereas an index of zero does not get the same treatment.

But also, again: Zig does not use malloc for its ASTs, as far as I know. It uses lists and indices. I haven't literally read the code lately myself, but I would be surprised if they went back to malloc'ing individual nodes.

[nicoburns]

> In extremely high performance code you use different data structures and algorithms and change your approach to memory allocation.

It's worth noting that the reason Rust doesn't include support for custom memory allocation patterns like Zig does has nothing to do with memory safety. It's more of a historical accident that it just wasn't something that was prioritised early in the projects history and is now hard to change.

[steveklabnik]

It's also important to note that Rust as a language does not know about the heap at all, it is purely a library concern. This means that "doesn't include support for custom memory allocation patterns" is purely talking about standard library data structures, and if you need a ton of performance, you're probably going to be writing your own anyway.

It will be nice when the Allocator trait stabilizes so that the ecosystem can coordinate on making this stuff pluggable, but that's not a direct blocker for getting things done if you need to do things today.

[narnarpapadaddy]

No disagreement in principle; in practice taking that approach in Zig is safer than in Rust, because in Rust it’s “all or nothing.”

[kibwen]

The interfaces to custom allocators are more idiomatic, standardized, and normalized in Zig, but there's nothing systematically unsafer about the Rust equivalent. You can use custom allocators all you want in Rust, as long as you're okay using third-party crates like Bumpalo.

On that topic, worth mentioning that Rust's long-awaited `Allocator` trait is perilously close to stabilizing; watch for https://github.com/rust-lang/rust/pull/157428 to be merged, then the stabilization PR to progress here: https://github.com/rust-lang/rust/pull/156882