[pizlonator]

(Sorry for talking about my personal project again y’all.)

What I want folks to get out of the Fil-C thing is that the whole notion that C is a memory unsafe language and Rust (or any other language) being safer is a subtle thought error.

The truth is: C as it is currently implemented is unsafe, and implementing it in a safe way is possible but we choose not because that would make it slower and use more memory.

The argument for using Rust instead of a memory safe implementation of C is all about performance. And that’s not a bad argument! But we should be honest about it.

(It’s also annoying that this DARPA page mentions some kind of consensus. There’s no consensus on Rust being the safe bet. Lots of orgs are going with alternatives, like Swift, Zig, C++-with-changes, or other things.)

[woodruffw]

> What I want folks to get out of the Fil-C thing is that the whole notion that C is a memory unsafe language and Rust (or any other language) being safer is a subtle thought error.

I think what you’re doing with Fil-C is cool and interesting. But I think you’re talking past a large part of the “memory safety” audience with this framing: most people don’t want “memory safety” qua uncontrolled program termination, they want memory safety qua language semantics that foreclose on (many, if not all) things that would be memory-unsafe.

Or in other words: it’s great to be able to effectively be able to forklift C onto a managed runtime, but most people really want to be able to write code that doesn’t crash at all because of expressive limits in C’s semantics. “Memory safety” is a convenient term of art for this.

[pornel]

Another aspect is that the majority of projects that keep using C, do it specifically to maximize performance or low-level control (codecs, game engines, drivers, kernels, embedded).

For such projects, a GC runtime goes against the reason why they used C in the first place. Rust can replace C where Fil-C can't.

A technically memory-safe C with overhead is not that groundbreaking. It has already been possible with sandboxing and interpreters/VMs.

We've had the tradeoff between zero-overhead-but-unsafe and safer-but-slower languages forever, even before C existed. Moving C from one category to the other is a feat of engineering, but doesn't remove the trade-off. It's a better ASAN, but not miraculously fixing C.

Most projects that didn't mind having a GC and runtime overhead are already using Java, C#, Go, etc. Many languages are memory-safe while claiming to have almost C-like performance if used right.

The whole point of Rust is getting close to removing the fast-or-safe tradeoff, not merely moving to the other side of it.

[pizlonator]

There are so many programs written in C or C++, but not in any of the languages you cite, that run just fine in Fil-C.

The thing that is interesting about Fil-C isn’t that it’s a garbage collected language. It’s that it’s just C and C++ but with all of the safety of any other memory safe language so you can have a memory safe Linux userland.

Also, Fil-C isn’t anything like ASAN. ASAN isn’t memory safe. Fil-C is.

[blub]

Many people know and like C. Many companies have access to plenty of C talent, but no Rust talent.

These are two great reasons to try Fil-C instead of Rust. It seems that many Rustaceans think in terms of their own small community and don’t really have a feel for how massive the C (or C++) universe is and how many different motivations exist for using the language.

[array_key_first]

I agree, but the converse is also true and is where the value of this DARPA grant lies:

There's a lot of legacy C code that people want to expand on today, but they can't because the existing program is in C and they don't want to write more potentially unsafe C code or add onto an already iffy system.

If we can rewrite in Rust, we can get safety, which is cool but not the main draw. The main draw, I think, is you now have a rust codebase, and you can continue on with that.

[pjmlp]

Many project do keep using C out of cultural and human reasons as well, they could have long written in a memory safe language, but goes against their world view, even when proven otherwise.

[pizlonator]

I’m not inventing a definition for memory safety out of thin air, so I think there’s just a tendency to conflate Rust’s static checking with memory safety.

Rust’s most important safety mechanism is panicking on out of bounds access. OOBA’s are the thing that attackers most want to do, and Rust, Fil-C, and almost all of the other memory safe languages solve that with runtime checking.

In short, I’d say what Rust gives you is static checking of a bunch of stuff and also memory safety. Fil-C gives you memory safety (and Fil-C’s memory safety goes further than Rust’s).

[duped]

I would disagree, because people don't really care about memory safety for security reasons as much as I think gets talked about.

The most important safety mechanism in Rust is the borrow checker. It solves so many real problems that people willingly put up with the steep learning curve because that's easier than debugging data races and use after free.

Bounds checking is nice but it's not interesting or compelling enough to talk about as some ungodly super power of safety and security.

[woodruffw]

> OOBA’s are the thing that attackers most want to do, and Rust, Fil-C, and almost all of the other memory safe languages solve that with runtime checking.

On browsers and other high-performance codebases? I would have guessed UAFs and type confusions would be higher on the attacker priority queue for the last 15 years. Rust prevents those statically.

[pizlonator]

My data comes from OSes as a whole not just from browsers or high performance codebases.

[blub]

“ most people really want to be able to write code that doesn’t crash at all because of expressive limits in C’s semantics. “Memory safety” is a convenient term of art for this.”

Rust doesn’t rely only on compile-time checks, so I find your claim dubious. Actually, the dynamic checks done by Rust are presented as useful features in themselves.

The memory safety audience and most people seem like arbitrary groups defined to support claims which aren’t really supported by strong evidence.

The biggest improvement in memory safety in the past decades wasn’t thanks to Rust, but languages like Python, Java or C# which rely heavily on dynamic checks. The goal is to catch bugs and avoid security vulnerabilities.

Taking standard C code and achieving the above would be amazing and much more useful than rewriting something in Rust.

[woodruffw]

I didn’t say it relies only on compile time checks. Where are you getting that from?

The point about expressive semantics is that you can foreclose on vulnerabilities statically, not that everything becomes a static check. That’s why I mentioned UAFs and type confusions in the adjacent response below.

> Taking standard C code and achieving the above would be amazing and much more useful than rewriting something in Rust.

As others have pointed out, you can’t do this without biting one of two bullets: you either change the language semantics to resemble something like Rust, or you add instrumentation that turns C into a managed language. Either of these forks is great, but neither is something you can do transparently: the former means breaking existing C code, and the latter means excluding a large percentage fraction of the world’s C code (where the absence of a managed runtime is a hard requirement).

Fil-C bites the second bullet, under the theory that there’s a lot of userspace code that isn’t extremely performance sensitive. And that’s probably true, but I also think it misses the larger trend: that a lot of userspace code is abandonware, and that there’s sufficiently more interest in rewriting it than maintaining it. For that reason alone, I think we’re going to see more userspaces in safe compiled languages, not unsafe ones that have been instrumented into safety.

[pizlonator]

> Fil-C bites the second bullet, under the theory that there’s a lot of userspace code that isn’t extremely performance sensitive. And that’s probably true, but I also think it misses the larger trend: that a lot of userspace code is abandonware, and that there’s sufficiently more interest in rewriting it than maintaining it.

Is there sufficiently more interesting in rewriting it, really? I know that there are a handful of userspace utils being rewritten in Rust, doing that is labor-intensive, and causes regressions (in perf and behavior), which then causes more labor (both for folks to deal with the fallout on the receiving end and to fix the regressions). And for every tool that has a rewrite in flight, there are maybe 100 others that don't.

[tialaramex]

Fil-C is going to usually be a huge perf regression right? Sometimes I won't care, sometimes I will and this will vary between users with only some commonality.

I think any software on the "most people are annoyed" list will get a rewrite. All the tools where people cared specifically about perf anyway are already rewritten or being rewritten because there are so many performance opportunities. The out-of-box grep sucks compared to ripgrep for example.

It will suck to be someone with no programming ability for whom a tool most people aren't annoyed by is too slow with Fil-C for their niche use. But that's not so different from how it sucks when your local government forgets deaf people exist, or that time we made everything use touch screens so if your fingers don't "work" on a touch screen now you're crippled with no warning.

[woodruffw]

I think so: you're 100% right about the labor, but OSS isn't a rational labor market :-). People RIIR not because it's easy, but because they find it more fun than the alternative.

(If it was, we wouldn't have dozens of tools that all NIH the same thing.)

[AlotOfReading]

There's plenty of reasons to prefer Rust to a memory-safe C. For example, I work on safety-critical code. That means that I have to review and affirm "This code implements my design". Making C memory-safe doesn't make the semantics of all UB defined or appropriate.

Safe rust doesn't have UB, and validating unsafe rust isn't meaningfully different than doing it for C++, except that it's limited to tiny leaf functions with well-defined interfaces where a code reviewer can challenge you.

[pizlonator]

> Making C memory-safe doesn't make the semantics of all UB defined or appropriate.

Fil-C largely gets rid of UB.

There are exceptions left but they’re all fixable.

The only reasons why C has UB are:

- lack of memory safety (Fil-C fixes this)

- policy (Fil-C mostly fixes this but there are still a small number of remaining issues, which I intend to fix).

Fil-C is super comprehensive in how it fixes C. I bet it’s more comprehensive than you’d expect.

The reason not to use Fil-C in safety critical systems is that some operations have varying cost and there’s a GC. The GC is concurrent so you might be able to make it suitable for safety critical hard realtime stuff; it would just be a lot of work and probably not worth it. Rust is better than Fil-C for hard real time stuff for that reason.

[AlotOfReading]

It's on my free-time backlog to spend more time with Fil-C, so I'm not disagreeing from lack of interest.

Most of annex-J is unrelated to memory safety. No, C has explicit UB because there wasn't a defined behavior that made sense to codify in the standards process. Signed overflow, invalid printf specifiers, and order of evaluation for example. I assume Fil-C doesn't fix things like uninitialized memory or division by zero either.

Wasn't really getting into the GC because that's "just" an engineering issue, rather than a structural issue with the approach.

It'd be great to not only terminate on detecting these issues as Fil-C does, but prevent them from happening entirely.

[pizlonator]

Fil-C absolutely does fix uninitialized memory.

It’s on my list to solve division. It’s easy to do and also not super important for the security angle that I’m addressing. But with doing precisely to provide clarity to these kinds of discussions.

I’ve mostly tackled signed overflow. I’ve fixed all the cases where signed overflow would let you bypass Fil-C’s own bounds checks. It’s not hard to fix the remaining cases.

In short: any remaining UB in Fil-C is just a bug to be fixed rather than a policy decision.

The reason why C has UB today is policy and memory safety.

Its a goal of Fil-C to address memory safety violations by panicking because:

- That’s the most secure outcome.

- That’s the most frantically compatible with existing C/C++ code, allowing me to do things like Pizlix (memory safety Linux userland)

[junon]

How does Fil-C "fix" uninitialized memory?

[tialaramex]

I assume under the same "memory safety" rationale it just zeroes the RAM. That's "safe" and compatible with C.

In a good language this mistake is caught at compile time, like in Rust, the compiler says "Hey, I don't see how this variable is initialized before use" and you slap your forehead and fix it. But zeroing everything is technically safe.

For the Casey "hand made" Muratori type zeroing might even seem like a better idea. It's cheap, it means now your code compiles and executes, who cares about correctness?

[pizlonator]

By initializing it to zero.

[scottlamb]

> The argument for using Rust instead of a memory safe implementation of C is all about performance. And that’s not a bad argument! But we should be honest about it.

I think it might also be that people mostly consider languages from the perspective of what they'd write a greenfield codebase in. And if I'm gonna pay the GC cost, I'd much rather work with language/library body that doesn't have tons of ownership idioms that are basically irrelevant with a GC. So it's not Fil-C vs Rust directly; it's Fil-C vs Go (Go wins), then Go vs Rust (more interesting decision).

But for existing large C projects that are ancient enough that running on modern hardware far outweighs the new GC cost, and that are mature/minimally maintained so the development costs for a rewrite/translation won't ever pay off in future maintenance, I think Fil-C is an intriguing option.

btw, IMO Rust has more going for it than just the borrow checker.

[pizlonator]

I think what you’re saying is true for the kind of software where it isn’t just greenfield but also doesn’t have to make deep use of preexisting system dependencies.

Like, say you’re writing a port daemon. Then your argument holds! And that’s admittedly a big category since it includes databases, application servers, and a lot of other stuff.

But say you’re writing an app that talks to systemd and your desktop shell and renders things and has a UI. If you do that in Go or Rust, you’re pulling in a ton of unsafe dependencies. But the Fil-C story is that you just compile all of that stuff with Fil-C and so there are no unsafe dependencies.

By the way, that’s not a fantasy but like mostly a reality. I have a Linux distro where all of userland is compiled with Fil-C and it works great!

Now you might say, why not rewrite all the deps in Go or Rust? The funny thing about that is you sort of can’t do that today since neither of those languages has dynamic linking. Fil-C supports dynamic linking just fine.

[woodruffw]

> Now you might say, why not rewrite all the deps in Go or Rust? The funny thing about that is you sort of can’t do that today since neither of those languages has dynamic linking.

I don’t know about Go, but that’s certainly not true for Rust. Dynamic linking is the norm in Rust; you can run `ldd` on any stock-built binary to see that it dynamically links to glibc (or libSystem or similar).

(It’s also common for Rust libraries to be distributed as shared objects with C APIs/ABIs — to my understanding, this is how virtually every incremental adopter of Rust has chosen to adopt it.)

[pizlonator]

When you do that, the ABI boundary around your shared library is unsafe.

So if you rebuilt userland with Rust and had the same layout of shared libraries then you’d have a massive amount of unsafe code

[woodruffw]

> So if you rebuilt userland with Rust and had the same layout of shared libraries then you’d have a massive amount of unsafe code

Sort of, but I think that’s misleading; you’d have a massive amount of safe code, scaffolded with unsafe ABI boundaries. That’s a problem, but it’s not really the same kind of attacker or fault surface as unsafe code in the libraries themselves.

It’s also a wholly solvable problem in the sense that Rust could define a stable non-C ABI. There just hasn’t been an extraordinary need for that yet, since most people who want to build Rust code into shared objects do want a C ABI.

[Splizard]

> neither of those languages has dynamic linking

what do you mean by this?

[kbolino]

Neither Go nor Rust define an ABI for themselves. They can use dynamically linked C code just fine but cannot link their own language with all (or even a reasonably broad subset) of its own features. This is not as bad in Rust, where you can fall back to the unsafe C ABI while still having safe Rust on either side, whereas Go cannot have two instances of its runtime active in the same process.

Compare this situation with Java/JVM or C#/.NET, both of which define their own ABIs albeit not for native machine code, or especially Swift, which defines its own native-code ABI.

[adastra22]

Does your thing add linear types, borrow checking, and remove implicit conversions from C?

[pizlonator]

Definitely not. Then it wouldn’t be a C implementation.

Fil-C is so compatible that I have a Linux userland compiled with it https://fil-c.org/pizlix

Think of Fil-C as Java-ifying C an C++

[adastra22]

I don’t see the relevance to this discussion then.

[NobodyNada]

The discussion is about a project for securing legacy code by machine-translating it to Rust. Fil-C is an alternative way to secure legacy code using a different C implementation. I think that's highly relevant to the discussion.

[adastra22]

The kind of errors being protected against are totally different though.

[pizlonator]

Fil-C protects against a superset of the errors that Rust protects against. It just does it dynamically.

And more comprehensively. There’s no `unsafe` statement in Fil-C. There’s no need for it since dynamic checking is vastly more precise.

[pizlonator]

You’re confusing memory safety with Rust’s specific flavor of static checking.

Totally not the same thing.

Like, notice how DARPA is in no hurry to have folks rewrite Java or JavaScript in Rust. Why? Because even JavaScript is memory safe. It’s not achieving that with dynamic checking rather than static checking.

[NobodyNada]

> The argument for using Rust instead of a memory safe implementation of C is all about performance. And that’s not a bad argument! But we should be honest about it.

This is half of it; the other half is that there's more to writing correct code than just memory safety.

Rust was not initially designed as a memory-safe "C alternative" -- it started as a higher level language with green threading and garbage collection. Borrow checking, affine types, and all that were introduced not to "solve" memory safety, but to reduce logic bugs in complex, highly concurrent code by enabling programmers to statically check application invariants using the type system. Once it became apparent that this system was powerful enough to express the semantics of full memory safety without garbage collection, GC was removed from the core of the language, and Rust became what it looks like now.

The point is, I think focusing on "memory safety" seriously undersells Rust. Rust is aimed at giving you tools to statically verify arbitrary correctness properties of your program; and memory safety is just one example of how the standard library uses the available tooling to prove one particularly useful property. But Rust's "safety" protects you from any logic bug that you define to be a safety issue.

Additionally, this means that you're not restricted to one particular runtime or environment's ideas of memory safety -- you can write or bring in unsafe code to define your own requirements. This is very important to me -- I work on low-level driver and RTOS code for microcontrollers, and am commonly doing very unsafe things with task switching, interrupts, DMA, memory layout and pointer casting tricks, inline assembly, hardware MMIO, power management, etc. Rust fits fantastically in this niche, because I can use the type system to write out all the preconditions that external users must uphold when interacting with my driver code.

I think Fil-C is an excellent project, and is far more practical and realistic than TRACTOR for the problem of securing legacy code. But I also see it as largely orthogonal to Rust -- I can't see many situations where someone would choose Fil-C for greenfield code over something like Java or Go. Rust has safety, performance, and interop advantages over GC'd languages; and safety advantages over other C-like languages; so it's a great choice for new projects, but it's not gonna help you with all your old code like Fil-C will.

> It’s also annoying that this DARPA page mentions some kind of consensus. There’s no consensus on Rust being the safe bet.

The "consensus" the article talks about is not Rust, it's that "relying on bug-finding tools is not enough." Swift, C++-with-[the right]-changes, and Fil-C all would fall within that consensus (of needing some sort of guaranteed safety); only Zig is something of an exception in your list.

[hluska]

The article doesn’t say the consensus was reached around Rust. That paragraph was talking about memory safety and so were both paragraphs around it. Rust isn’t even mentioned for two paragraphs after the part that annoyed you so much. But the first four paragraphs were all about memory safety.

[safercplusplus]

As long as we're plugging our projects, I'll mention the scpptool-enforced memory-safe subset of C++. Fil-C would be generally more practical, more compatible and more expedient, but the scpptool-enforced subset of C++ is more directly comparable to Rust.

scpptool demonstrates enforcement (in C++) of a subset of Rust's static restrictions required to achieve complete memory and data race safety [1]. Probably most notably, the restriction against the aliasing of mutable references is not imposed "universally" the way it is in (Safe) Rust, but instead is only imposed in cases where such aliasing might endanger memory safety.

This is a surprising small set of cases that essentially consists of accesses to methods that can arbitrarily destroy objects owned by dynamic owning pointers or containers (like vectors) while references to the owned contents exist. Because the set is so small, the restriction does not conflict with the vast majority of (lines of) existing C++ code, making migration to the enforced safe subset much easier.

The scpptool-enforced subset also has better support for cyclic and non-hierarchical pointer/references that (unlike Safe Rust) doesn't impose any requirements on how the referenced objects are allocated. This means that, in contrast to Rust, there is a "reasonable" (if not performance optimal) one-to-one mapping from "reasonable" code in the "unsafe subset" of C++ (i.e. traditional C++ code), to the enforced safe subset.

So, relevant to the subject of the post, this permits the scpptool to have a (not yet complete) feature that automatically converts traditional C/C++ code to the safe subset of C++ [2]. (One that is deterministic and doesn't try to just punt the problem to LLMs.)

The problem isn't dedicating public resources to trying to getting LLMs to convert C to Safe Rust after investments in the more traditional approach failed to deliver. The problem is the lack of simultaneous investment in at least the consideration and evaluation of (under-resourced) alternative approaches that have already demonstrated results that the (comparatively well-funded) translate-to-Rust approach thus far hasn't been able to.

[1] https://github.com/duneroadrunner/scpptool/blob/master/appro...

[2] https://github.com/duneroadrunner/SaferCPlusPlus-AutoTransla...

[heavyset_go]

I'm genuinely curious where Swift is being used outside of macOS/iOS apps

[iberator]

Hey. Thanks for sharing. Never heard about Fil-C before. Looks interesting