[tialaramex]

Steve is too modest about the tests by the way. Lots of the conformance testing for Rust's compiler happens automatically. Herb mentions Perennial https://www.peren.com/ and Plum Hall https://plumhall.com/ for C++ and Steve says "I'm not saying that functions as a conformance test suite" when it comes to Rust's own compiler tests. But, while the Rust project itself does not perform conformance testing, Ferrous Systems does and so yeah, analogous conformance test suites exist and are run automatically.

So when Herb says "And in Rust's case, that's going to need to be built out" if the interview was taking place five years ago that's an important point. But it wasn't five years ago, it was only a few weeks ago.

[bregma]

> analogous conformance test suites exist and are run automatically

I'm not clear on how this is even possible. "Conformance" for C++ means adherence to the published ISO/IEC 14882 standard. In the case of Rust, it's "this is what the compiler does this week". Sure, a third party has built out a CI system for Rust and uses it provide evidence that Rust does what Rust does, but that's nothing similar to having evidence any random toolchain conforms to an accepted, published, international standard of what the C++ language does.

When it comes to providing test evidence for functional safety (for example ISO 26262 or IEC 61508) I can point to Perennial or Sold Sands results and say with confidence that my toolchain does what it says on the C++ box. When it comes to Rust, I can point to Ferrous and say "the core Rust language when built using rustc does what the rustc developers claimed it should do last week, excluding any crates or libraries." To which claim are you going to trust the lives of you and your loved ones?

They aren't analogous at all. There is just reams of marketing spin here and eventually it's going to kill someone.

[lifthrasiir]

Here the conformance refers to the Ferrocene language specification [1], which is being developed to be something akin to ISO/IEC 14882 for Rust in the future. In fact, this specification and the Ferrocene toolchain is meant to be ISO 26262 or IEC 61508 qualified, so it directly addresses your concern.

The comment posted by jamesmunns will do much better job in describing the exact details than mine (both comments happened to be posted within 20 seconds, huh), so I'll just add that standards are meaningless by their own. Quite a lot of people criticizing Rust for the lack of formal standards seem to assume that such standards will immediately improve something, but that's never true. Standards are meaningful only when they are maintained and can be enforced; many historical standards failed to catch up and fell into the oblivion for this reason. It doesn't even matter much whether the specification is written in "formal" prose [2]. I feel they are missing the fundamental reason why language standards can be beneficial in principle.

[1] https://spec.ferrocene.dev/

[2] Which is never "formal" in the mathematical sense. (Mathematically formal language specifications are quite rare, examples include SML and WebAssembly.) In fact, such formal writing is slightly better than informal writing only because such writing also aims to be unambiguous, and the formality itself is not that important.

[nurettin]

> Quite a lot of people criticizing Rust for the lack of formal standards seem to assume that such standards will immediately improve something

Formal standards aren't just for providing a spec to be used in alternative implementations or somehow proving that the syntax is logically sound.

They are also the process of international representatives vetting for new additions to the language or breaking backwards compatibility which is what I would be criticizing the lack thereof.

[lifthrasiir]

Most "international" standards are international only because the corresponding standard organization is ultimately operated by multiple nation-states. In reality the organization just appoints some experts to write the standard and brand it with their blessing, so the standardization process is hardly international. For language standards, those experts typically include original designers and whoever wanted to standardize the language. (Otherwise why would anyone want to join this tedious process?) A diverse group of experts is very rare for such standards.

It is another myth or fetishism that international standards have some inherent mechanism to ensure the compatibility. Such quality can be only maintained when standard editors take care of that and there is a comparable industry support. Missing any one, the compatibility can be easily broken, and conversely any specification---formally written or not---can maintain the compatability when both are satisfied.

[steveklabnik]

The Rust Project also has international representatives vetting new additions and ensuring backwards compatibility.

[shakow]

Are those standards really useful though? I'm under the impression that Perl/Ruby/Python/etc. have done well enough without.

[steveklabnik]

Hilariously, Ruby actually has an ISO standard, and it is absolutely useless.

[lifthrasiir]

It has been fast-tracked from the Japanese standard JIS X 3017, and I think there was a strong incentive to standardize due to Ruby being the only current mainstream language originated from Japan. (I recall even the Ruby Association [1] is supported by Matsue, where Matz was born.)

[1] https://www.ruby.or.jp/en/

[jamesmunns]

This is entirely incorrect, Ferrocene manages a specification of a subset of the language here: https://public-docs.ferrocene.dev/main/specification/index.h..., including things like a traceability matrix that links the specification to the implementation and the verification: https://public-docs.ferrocene.dev/main/qualification/traceab...

For each release of Ferrocene, this is kept up to date, and the same as C or C++, what is specified and stable can be relied upon, and implementation details are implementation details, the same as it would be if you switched from LLVM w/ SolidSands' SuperTest suite to IAR or GreenHills' toolchains which may have varying impl details but still maintain conformance with the specification.

The majority of safety critical teams will snapshot a single toolchain for the entire development lifecycle (sometimes updating if necessary, very rarely), but Ferrocene is releasing updates that are approaching the full Rust cadence (IIRC they've discussed going to every other release, so once every 12 weeks vs Rust's 6 week cadence), with all of the verification required to ensure the specification is still complete, and all tests are passing.

There's still work to specify and test more/all of the core/alloc/std library components, as well as third party crates, but from a toolchain perspective, it is much closer than you are giving them credit for. Unlike many proprietary C/C++ toolchains or verification suites, the majority of safety justification artifacts are publicly browseable here: https://public-docs.ferrocene.dev/main/index.html.

(I am a former founder of Ferrous Systems, and one of the people that pushed for the Ferrocene project to happen, but haven't worked there for a couple years and have no monetary stake in them anymore - I think they are just still doing the right thing, and doing it well.)

[tialaramex]

"What it says on the C++ box" is indeed enough for (parts of) 26262

And that's a problem, because what it says is vague and not very helpful for the problem we actually wanted to solve. It's enough by fiat - the people responsible for 26262 wanted to be allowed to write C++, so the bar has to be low enough to be cleared by C++. This problem grows in magnitude as the software in a safety of life critical system grows.

To be clear: I don't think either of these general purpose programming languages is appropriate for use in safety of life applications. I think a provably correct program ought to be table stakes - But, a bar set low enough that you can write C++ is always going to be low enough for Rust to just step over it like it's barely even there.

[steveklabnik]

If I’m being too generous in this discussion (and I was trying to be), saying “whatever the compiler does this week” is far too harsh. The Rust Project has a commitment to stability that they’ve followed through with for almost a decade now. The amount of times my existing code has broken has been very, very low.

[pjc50]

Conformance is meaningless so long as the spec contains the words "undefined behavior" anywhere.

[myworkinisgood]

C++ leadership are blinded by their own success and have no information of outside world.