[rtfeldman]

> I think that for the hot binary patching / code reloading features, yes, that is going to need unsafe. But for regular old "producing an executable" compilation? Emitting machine code isn't the part that requires unsafe. The language's runtime is a more likely site to find unsafe.

Agreed! Emitting machine code is not unsafe, since it's just writing bytes down - it's only once you execute that machine code that there's potentially unsafety. The reason I said "a big part of the job" is that in practice a lot of compilers both emit machine code and execute it - but you're totally right that it's not a requirement that a compiler do both.

In addition to the examples you gave (hot binary patching/code reloading, language runtime, etc.), others would be things like evaluating userspace code at compile time (e.g. const fn in Rust, or in Roc any expression that could be hoisted to the top level), running tests and inspecting their output to decide what to display to the user, etc.

Those are the types of things I had in mind when I wrote that.

[steveklabnik]

I am disappointed you're downvoted, Richard. This is a fine reply, and I hope you know that a minor quibble with a single line in the post doesn't mean that I think it's a bad one overall. (EDIT: a few minutes later, the parent comment is no longer grey.)

I also think it's a good thing that you wrote the post in general, when I saw it pop up I was like "oh, of course, this post should exist!" I'm surprised I didn't think about it earlier.

> evaluating userspace code at compile time

Usually this would be done via an interpreter, so I'm not sure that it really requires unsafe either. If you are literally executing machine code, sure, but const fn in Rust and constexpr in C++ and many other languages do not do that, as it causes a number of problems (for example, cross-compilation).

[rtfeldman]

Also a good point! TIL that Rust and C++ use interpreters for const, although of course that wouldn't work for running tests. Then again, in the specific case of Rust I believe rustc only compiles the tests and then something else like Cargo executes them. Of course, as I noted elsewhere, if rustc emits machine code and then cargo immediately executes it, there's the same opportunity for end user memory being corrupted (due to miscompilation) as if rustc and cargo shared a code base.

By the way, I thought your question was totally reasonable - my first thought reading it was "Oh yeah I wasn't trying to say that writing bytes is unsafe, I definitely should have worded that differently."

[steveklabnik]

Cool, I'm not sure that people know that we know each other and have some deeper mutual understanding. :)

> although of course that wouldn't work for running tests.

Why not? Unless you mean in the cross-compilation case, in which yeah, to run the compiled tests you'd need an emulator.

> in the specific case of Rust I believe rustc only compiles the tests and then something else like Cargo executes them.

It doesn't have to be Cargo, but yes, rustc produces executables for the tests, and you have to then run them.

> there's the same opportunity for end user memory being corrupted (due to miscompilation)

I agree for sure that the safety of the outputted binary is completely distinct from the safety of the compiler itself.

I think the reason that this framing specifically (in the post and in this comment) strikes me as odd is that "requires unsafe code" sort of implies that you need to use unsafe to fix the unsafety of the outputted binary. That just isn't the case. Of course, this is a serious bug that needs to be fixed, but there's just something about "doing memory unsafe things" in this area that like, I think can be a little mis-leading, even if that's not intentional. But I am going to sit with this and think about it, regardless, because I am not sure that my gut reaction here is completely accurate.

(And, hilariously, looking over some work my agents did on my compiler last night, they fixed some mis-compilations that occurred, entirely in safe code. I bet that's also part of why I'm in this headspace at the moment, it's not like those fixes required dropping down into unsafe to fix either!)

[Liquid_Fire]

> if rustc emits machine code and then cargo immediately executes it, there's the same opportunity for end user memory being corrupted (due to miscompilation) as if rustc and cargo shared a code base

Your tests run in an entirely separate process from the compiler (and from cargo). This makes it very different from memory corruption in the compiler:

- The test process can only corrupt its own memory.

- You don't need "unsafe" to run tests. Just the ability to start another process.

- If you're cross-compiling, you wouldn't even be able to run the tests on the same machine (without emulation/compatibility layers)

Does roc run tests in the same process as the compiler?

[rtfeldman]

> Does roc run tests in the same process as the compiler?

We do for tests of pure functions, yes.

> Your tests run in an entirely separate process from the compiler (and from cargo).

That's a great point and a relevant distinction, although Rust tests can run arbitrary I/O, so it's not like having them be in a separate process means memory corruption is harmless! :)

[Liquid_Fire]

True, but not sure how it's relevant - you don't need memory corruption for the test to perform arbitrary I/O. And even if you trust the tests to not do anything bad, you don't need memory corruption for this to be an issue - any bug in the code emitted by the compiler could cause bad things to happen.

But all of this is ultimately completely unrelated to the concept of "unsafe". You can delete your home directory just fine in safe Rust/Go/Python/etc. You can write a compiler that emits broken code in the same languages; even in a 100% pure functional language with 0 side effects and a perfect bug-free implementation.

[minraws]

I would like to understand this more,

> rustc emits machine code and then cargo immediately executes it, there's the same opportunity for end user memory being corrupted (due to miscompilation) as if rustc and cargo shared a code base.

Cause this hasn't been true for me or for anyone maybe your definition of memory being corrupted is the not same as mine.

I am not even sure what you are trying to prove with this.

I appreciate the time and effort in building stuff like Roc I don't use it but this comment and the article feel like...

Oh some guy said Zig not nice because memory safety so here, a post why memory safety doesn't exist because we have to do memory unsafe things sometimes and so everything is memory unsafe already, so maybe it doesn't matter.

I get the energy that we are going for seeing useless claims and wanting to push back but I think the article deserves a clearer part 2 where you elaborate on your thoughts about stuff maybe even get it peer reviewed a bit before posting or maybe don't I guess we could use more raw thoughts in the post AI age.

Either way I appreciate someone trying to put forward their own thoughts and explain problems with a different perspective.

[samatman]

Scenario A: A program has a memory vulnerability. That's bad, and the reason it's bad: attacker-controlled data can potentially trigger this vulnerability, which could even lead to privilege escalation.

Scenario B: A program writes machine code in an executable region of memory, and the code has a memory vulnerability. That's bad, and the reason it's bad: attacker-controlled data can potentially trigger this vulnerability, which could even lead to privilege escalation.

Scenario C: A program write machine code to disk, which is then read into memory and executed. That's bad, and the reason it's bad: attacker-controlled data can potentially trigger this vulnerability, which could even lead to privilege escalation.

[dwattttt]

Or scenario D:

A text editor stores a URL in a location on disk, and another component then fetches and executes that with its current privileges.

[fragmede]

Or in this day and age, that component is an AI agent, and it executes with the things stored in its memory, and is abused into exfiltrating data.

[minraws]

How is that a memory safety issue in Rust compiler? What point are we making here. Am I just too dense to understand, is how the hell is that a need unsafe issue? Or anything to do with compilers???

[jibal]

TFA discusses this in detail. The point is memory errors in the generated code. The generated code is an output the compiler. Thus the OP counts these against the compiler--after all, that's his project, those bugs are real, and those bug reports must be dealt with. His point is that the Rust borrow checker doesn't help at all with these bugs, and that matters when weighing the pros and cons of writing the compiler in Rust vs. Zig -- the languages are a wash for that category of error.

I do think this is a bit off, because miscompilations can produce all sorts of errors, not just memory errors, so it doesn't make much sense to categorize them as memory errors -- they are logic errors in the compiler. But the basic idea holds -- the choice of implementation language doesn't matter.

[Joker_vD]

> Usually this would be done via an interpreter, so I'm not sure that it really requires unsafe either.

Well, I personally have written a const-expression evaluator that actually reuses the rest of the compiler: it compiles the expression in the current environment with some specific adjustments to the codegen settings, launches the temporary executable and gathers its output... frankly, it's more hassle than it's worth compared to writing a separate const-expression interpreter. Plus, of course, it also runs slower since most constant expressions are usually pretty trivial.

[jibal]

It depends somewhat on the complexity of the language.

Nim uses an interpreter but Nimony, which is destined to becomes Nim 3.0, uses your approach. It will be interesting to how hassles and performance play out there and whether they keep the compile-and-run approach or go back to an interpreter.

[psychoslave]

Side concerne, but reading the post initially I thought it was about Rocq (formerly Coq). Maybe both team could coordinate and find a way that everybody agree on to avoid any confusion?

https://en.wikipedia.org/wiki/Rocq