Hacker News new | ask | show | jobs
by hardware2win 1280 days ago
I hope there aint gonna be ecosystem fragmentation

When there are a fews compilers with significant market share then developers are those who lose, lose that handy dev. experience of solid and consistent ecosystem

I hope it will be used only where necessary

5 comments

The gccrs developers have talked at length with Rust developers towards the same aim: they don't want to create fragmentation either. Among other things, the gccrs developers have said in multiple places both public and private:

- They aren't going to fork or extend the language; they'll work with the normal Rust language evolution process.

- They're treating rustc as the reference for correct Rust. This doesn't mean there can't be bugs in either rustc or gccrs, and rustc will deeply appreciate bug reports found through this level of inspection of its behavior, but nonetheless this is a stated goal of the gccrs folks.

- They plan to explicitly provide messaging, potentially even in compiler error messages, to the effect of "if gccrs doesn't compile a crate that does compile on rustc, that's a bug in gccrs, not a bug in that crate; do not report it as a bug in that crate or ask crate developers to work around gccrs limitations".

So, I think the gccrs folks are doing everything they can to avoid fragmentation, short of the project not existing at all.

They don't have to extend the language to create fragmentation. Being far enough behind rustc is enough
> Being far enough behind rustc is enough

But rustc itself can be far enough behind rustc, as the MSRV debate shows.

You can already do that. Some libraries wilfully restrict their MSRV for backwards compatibility or distro compatibility (e.g. Debian stable provides 1.48).
No argument there, but I think the third point helps address that somewhat.
I think this is more of an attempt of making Rust compilable on platforms not supported by LLVM, not an attempt to replace the existing compiler. The GCC frontend, for instance, does not implement the borrow checker, so you should only use it to compile Rust code you know is correct according to the official Rust compiler.
That's rather rustc_codegen_gcc. gcc-rs is more for people who don't like LLVM and/or Rust.

https://github.com/rust-lang/rustc_codegen_gcc

A rust implementation for people who don’t like rust seems like a strange take.
Not too strange. Imagine the reverse: a C compiler written in Rust. There is an awful lot of software in C that I want to use, but I really dislike C itself. If I found some deficiency in C compilers and felt the need to write my own, I certainly would prefer to write it in Rust.

It would be a C implementation for people who don't like C.

It seems like an odd idea to release a Rust compiler without a borrow checker. Isn't there a high risk that it will confuse users into thinking they have the usual Rust guarantees at run-time?
This can be very useful for platforms where rustc is not available due to LLVM not supporting them. Take a project, build it with rustc, and if it builds, you know it passes the borrow checker. It is then safe to build it for your actual target with gcc. This would limit the outcry when some software adds a dependency on Rust, like the python crypto thing of a few years back, or the Linux kernel.
This will still use the official rustc frontend. For what I described above, this is not an issue, it would totally work. However, there are cases where having a pure gcc implementation (frontend + backend) is beneficial.

As others have mentioned, vendors have the standard practice to fork gcc for their hardware, but not necessarily rustc nor libgcc used by rustc_codegen_gcc. This means that rust will be possible on those platforms for free thanks to gcc-rs.

Another point is the bootstrapping of the compiler. rustc has a quite complex bootstrap process that depends on python, while afaik gcc only relies on shell scripts. It is much easier to port gcc than rustc to a new host platform. This is useful when you want to compile code directly on your target and don't want to rely on cross compilation. cross compiling is a tricky thing to do, and while rust fares pretty well there, as soon as it uses native C libraries, it all falls apart (e.g. try cross compiling a project that has dependencies on libss, libpq and the like). In such case, native compilation may be the only viable option. And most of the times, GCC is the first thing to be ported, so you would not need to wait for a rustc port.

Borrow checker is compile time. There is no reason to have multiple implementations.
Yup, it's compile-time, and gives you run-time guarantees.
I think the expectation for now is that people will develop an app with rustc and then use gcc to compile to targets that aren't yet supported by llvm.
This is eventually going to be a feature-complete compiler, targeting a specific rustc version. I believe the plan is to use polonius [1], presumably as an "optional" feature so they can build a stage 1 without it, use that to build polonius, then build the final compiler with it included.

[1] https://github.com/rust-lang/polonius

You are right, this much more ambitious than I though! Here is more information about the project, for the curious: https://rust-gcc.github.io/
Lacking a borrow checker is a temporary thing, no point reinventing the wheel. I think the intent is that eventually it'll use the same library as Rust (maybe Polonius, or perhaps plans will change)
No borrow checking is effectively a dialect of Rust?
Without a borrow checker, at best it can be considered a code generator for pre-certified Rust source code. Possibly useful, but you'd still need to have rustc in the loop.
That could be a feature. I'm making a joke, but it would be tempting sometimes when the program is correct but the borrow checker can't figure that out. The borrow checker and I don't get along all that well, even two years into the relationship.
I disagree. We need competition. I'm not using (even avoiding) LLVM based toolchains, and not touching Rust even with a 100 foot pole because of that.

On the other hand, I acknowledge that LLVM kickstarted GCC development and made it better in thousand ways better, agile and flexible-minded (in terms of developers maintaining it).

Similarly, having two Rust compilers, nudging each other to be better is a great way to make a language more general, widely accepted, resilient and more free & open. Also Rust well be much more stressed and tested as a result.

> I hope it will be used only where necessary

I hope both will be used widely, and make Rust a better language with much better accessibility.

> We need competition. I'm not using (even avoiding) LLVM based toolchains, and not touching Rust even with a 100 foot pole because of that.

I'm interested in this. Could you elaborate on why you are not using Rust? Because there is no competition in the language, or because there is not an alternate compiler, or because you don't like LLVM?

It's mostly a personal choice. Any piece of software I write is opened up, and licensed with GPL.

I don't want my software dependent on a non-GPL toolchain to ensure its long term sustainability. I also try to choose GPL licensed libraries, and include everything required to build my code (incl. external libraries) inside my repository.

In the end I want someone to be able to just clone the repo, and run make, or GCC, or what it needs to be run. So, the code is just dependent on the compiler and its standard libraries when obtained by someone.

Lastly, I find LLVM's supporters and general ecosystem slightly hostile towards GCC, and want to replace it anywhere and everywhere possible. I'm a free software advocate, and I prefer LLVM and GCC just to be interchangeable.

Hence I only use languages which GCC supports exclusively or has GPL compilers/interpreters. Working on HPC makes this easier for me.

I upvoted you. I have no idea why your comment received downvotes.

I responded in a similar way to a conversation about Julia less than a week ago.

For people who have trouble understanding this reasoning, a message from the GCC mailing list (https://gcc.gnu.org/legacy-ml/gcc/2014-01/msg00247.html) provides some of the explanation, which I reproduce here:

In the free software movement, we campaign for the freedom of the users of computing. The values of free software are fundamentally different from the values of open source, which make "better code" the ultimate goal. If GCC were to change from a free compiler into a platform for nonfree compilers, it would no longer serve the goal of freedom very well. Therefore, we had to take care to prevent that.

(See http://www.gnu.org/philosophy/open-source-misses-the-point.h... for more explanation of the difference between free software and open source. See also https://thebaffler.com/salvos/the-meme-hustler for Evgeny Morozov's article on the same point.)

The Clang and LLVM developers reach different conclusions from ours because they do not share our values and goals. They object to the measures we have taken to defend freedom because they see the inconvenience of them and do not recognize (or don't care about) the need for them. I would guess they describe their work as "open source" and do not talk about freedom. They have been supported by Apple, the company which hates our freedom so much that its app store for the ithings _requires_ all apps to be nonfree. (*)

The nonfree compilers that are now based on LLVM prove that I was right -- that the danger was real. If I had "opened" up GCC code for use in nonfree combinations, that would not have prevented a defeat; rather, it would have caused that defeat to occur very soon.

For GCC to be replaced by another technically superior compiler that defended freedom equally well would cause me some personal regret, but I would rejoice for the community's advance. The existence of LLVM is a terrible setback for our community precisely because it is not copylefted and can be used as the basis for nonfree compilers -- so that all contribution to LLVM directly helps proprietary software as much as it helps us.

The cause of the setback is the existence of a non-copylefted compiler that therefore becomes the base for nonfree compilers. The identity of that compiler -- whether it be LLVM, GCC, or something else -- is a secondary detail. To make GCC available for such use would be throwing in the towel. If that enables GCC to "win", the victory would be hollow, because it would not be a victory for what really matters: users' freedom.

If you think we ought to "compromise" on this point, please see http://www.gnu.org/philosophy/compromise.html.

The only code that helps us and not our adversaries is copylefted code. Free software released under a pushover license is available for us to use, but available to our adversaries just as well. If you want your work to give freedom an advantage, use the leverage available to you -- copyleft your code. I invite those working on major add-ons to LLVM to release them under GNU GPL version-3-or-later.

* If a binary is made from published source code, but you can't install your binary of a modified version of that source code, the binary is proprietary even if the source code is free. (See http://www.gnu.org/philosophy/free-sw.html.) A binary in Apple's app store may be made from published free source code, but under Apple's rules and Apple's DRM, the binary can't be free.

> I have no idea why your comment received downvotes.

I have to agree re: the parent. It's a perfectly fine opinion to hold, though it's one I don't share. I asked and he/she told me.

However, republishing an email by Richard Stallman, just like Richard Stallman, might seem a little excessive.

Why? I think it's because although just as it's fine to be an atheist, it always seems weird when someone brings it up at dinner parties. We were discussing the whys and wherefores of a new Rust compiler. I'm not sure any lengthy expressions of our licensing religion or lack thereof is a positive contribution.

The irony of Richard's decision and justification is how misguided it was (and is) on multiple fronts.

First, GCC as a whole has never been technically superior to the commercial competition. The reason why GCC was popular outside the GNU crowd and received as much funded development is that it was the best compiler across a range of popular platforms and had the best price, too.

Second, one of the main drivers behind the creation of clang has been the hostility to making the C/C++ frontend of GCC usable as a standalone tool. Richard explicitly wanted to force everyone to either deal with the GPL and his interpretation of what is derived work, or shell off the bucks for the Edision Design Group. This was a calculated choice as no one would want to invest the time to write another C/C++ frontend, right? He intentionally ignored all other use cases for C/C++ parser as a library (Language server, anyone? Refactoring tools?) and thereby actually hurt the user base on the GNU platform just as well. It can not be understated how disruptive clang was as a set of libraries outside the LLVM/clang compiler.

Third, GCC had as middle and backend compiler always was moderately accessible, but by structure somewhat harder to access for researchers or people wanting on special purpose tools like optimized code generation for computational kernels on matrixes.

GCC was forced to acknowledge the competition by finally getting a plugin interface in the GCC 4.5 days, but at that point, the cat was out of the bag.

>We need competition

There is - between langs.

Compiler engineers can use ideas from other lang's compilers.

C# e.g

I think having a GPL implementation of a language makes it more future proof. I don't want to be able compile some source only with "rustc-v_ancient-company_name-internal_fork-no_you_cant_have_its_source" version.

Programming languages with single compilers tend to be not used in my area of work and research (HPC and Scientific Computing). Hence, I'd rather have multiple compatible, yet independently implemented compilers, per language.

> I don't want to be able compile some source only with "rustc-v_ancient-company_name-internal_fork-no_you_cant_have_its_source" version.

Yeah, it's not like Google (and dozens of others) has/have an internal fork of Linux, which never gets merged back. I'm not certain the GPL butters any parsnips here.

> Hence, I'd rather have multiple compatible, yet independently implemented compilers, per language.

You identified it. The problem is compatibility, and again Linux is a good negative example where ostensibly we have a language spec and multiple compilers, but Linux for years would only compile on one compiler.

Multiple compilers and "competition" doesn't solve compatibility. It makes it more difficult. Although I'm sure there will be benefits, I'm not certain the GPL/GCC will make compatibility any easier. It hasn't so far.

> Yeah, it's not like Google (and dozens of others) has an internal fork of Linux...

Doesn't matter for completely internal stuff. My concern is public code requiring non-public toolchains to compile. I have experienced this enough during the decades. I don't want to fight with this again.

Google doesn't want to touch Linux for their devices anymore, so they're building Fuchsia, but let's not digress...

> Multiple compilers and "competition" doesn't solve compatibility. It makes it more difficult. Although I'm sure there will be benefits, I'm not certain GPL will make compatibility easier. It hasn't so far.

What GPL brings to the table is strict openness, not compatibility. I can share my code and say that "It builds right on this $GPLd_Compiler", and people can get it and build it. This is what I like to bring to the table for anything I release.

I also wonder whether people would be this reactive to this issue if $company announced a closed source compiler with strict rustc compatibility.

> What GPL brings to the table is strict openness, not compatibility.

I kinda wonder where this has been an issue with LLVM re: non-public toolchains, and public code where it wouldn't also be an issue with GCC. FWIW I definitely could see it being an issue in HPC/graphics/ML.

> I also wonder whether people would be this reactive to this issue if $company announced a closed source compiler with strict rustc compatibility.

Call me dense -- I guess I don't understand precisely what you're getting at. I don't think anyone would use a closed source compiler with strict rustc compatibility without a pretty big carrot. I think the issue here is strictly compatibility/divergence. Although I'm personally less of a fan of GCC/GPL/FSF, I think a new compiler is great so long as it doesn't (completely) ruin some of the nice things about a single implementation system, and why rust_codegen_gcc seems much more appealing.

Many Rust people don't want to live in the C/C++ compiler world, because they don't have to. "Oops this code won't build with GCC" is a battle they'd just as soon avoid.

>I don't want to be able compile some source only with "rustc-v_ancient-company_name-internal_fork-no_you_cant_have_its_source" version.

I dont think it is valid concern in $currentYear

Nowadays languages like e.g c# are partially created by community and foundations (.net foundation) on github

So not only isnt it fully tied to company, but also it is oss.

>Hence, I'd rather have multiple compatible, yet independently implemented compilers, per language.

I dont because it permanently decreases developer experience and the value added is not something that could not be achieved with single compiler

I mean if you see room for improvement then go ahead and make PR instead of creating yet another compiler with its own bugs, quirks and wtfs

> I dont think it is valid concern in $currentYear

I don't care about $currentYear. My concern is about $nextDecade earliest, and future doesn't look bright from my perspective.

> Nowadays languages like e.g c# are partially created by community and foundations (.net foundation) on github

A foundation created by Microsoft for a Microsoft's cash cow language on Microsoft's OSS entanglement platform. Yes.

> but also it is oss.

I'm not talking about OSS, I'm talking about Free Software.

> I dont because it permanently decreases developer experience and the value added is not something that could not be achieved with single compiler

This is not what happened with clang vs g++/gcc. Also we have Intel's, Microsoft's and Portland groups compilers side by side for decades.

> I mean if you see room for improvement then go ahead and make PR instead of creating yet another compiler with its own bugs, quirks and wtfs

I prefer implementing language correctly in a compiler to force others to the same, spec-compliant behavior.

Otherwise compilers and languages divert, and lack of alternatives make everything much more complicated in the long run.

>This is not what happened with clang vs g++/gcc. Also we have Intel's, Microsoft's and Portland groups compilers side by side for decades.

What? Cpps ecosystem is the most developer hostile ecosystem that Ive ever used

I wouldnt want to write in it even if my salary was multiplied by 1.5

A lot of compilers where all of them have different lang. features implemented, various bugs, quirks, wtfs, various perf. characteristics (e.g intel's generating faster code)

Long as hell compilation times. This is fundamental problem yet still that diversity didnt manage to solve it

Many package managers

And mediocre lang by modern standards on top of that. Minefield++ would fit it better.

>Otherwise compilers and languages divert, and lack of alternatives make everything much more complicated in the long run.

Theres still competition between langs

I bet Rust helped cpp improve more than gcc clang diversity

I also dont fully understand why you treat lang as a standard instead of product

It is already a concern with, for example, clang.
I’ve come to the conclusion that fragmentation is inevitable as a language gets popular.

People think differently, have different ideas for how to do things, need it to work for their particular use case, etc. Fragmentation and bloat are a result.

I am willing to bet it will happen rust as it gets used more and more. Then the treadmill will start again with a new language.

(Sorry if that’s jaded. I’ve been fighting with python build systems lately. What a nightmare)

>> I’ve come to the conclusion that fragmentation is inevitable as a language gets popular.

That's what language standards are for.

The Ferrocene project is working on standardizing Rust and creating a safety-critical version of the Rust toolchain:

https://ferrous-systems.com/ferrocene/

The Ferrocene Rust draft spec is out:

https://spec.ferrocene.dev/

As Rust continues to improve and be more widely used and adopted, there will be multiple implementations. As you've stated, it is inevitable as Rust gets popular. (For example, if Rust becomes popular enough to displace niches that C and C++ have occupied, when will Microsoft release a Rust toolchain? When will Intel release a Rust toolchain? And so forth.)

Standards help to better define what a Rust toolchain should do and what it means to be a conforming Rust implementation.

Standards are necessary and help of course. But the real power always lies with the implementations. There is no compliance without enforcement.

For example, C++ has a standard, but compliance is all over the place. Even Fortran has the same problem. The web also had a standards body (W3C) but eventually that was completely ignored and became irrelevant.

>> Standards are necessary and help of course. But the real power always lies with the implementations. There is no compliance without enforcement.

You are correct.

Compliance depends on the implementation developer / maintainer. If the standard exists and is agreed upon, at least users can petition the implementation developer / maintainer and say "Your implementation does not conform to section 3, paragraph 2, line 1 of the standard. The expected behavior is X, but your implementation does Y. Please fix it."

If a standard does not exist, then who is to say what the correct behavior should be?

If there is a standard, but no one follows it, then perhaps the standard needs to be adjusted. (Why is the standard not being followed? Were bad ideas standardized? Did the standards committee follow after one vendor's implementation to the detriment of other vendors' implementations? etc.)

Counter example: .net c#

One compiler for 99%, one package manager, one base classes liberary, one cli, etc.

One Microsoft Way.
It's certain that there will be fragmentation. There isn't any gnu project, which hasn't added its own extensions to standards or specs. Few examples: gcc front end for C/C++, bash, grep..

It means that crates.io will have crates which compile only with gcc and its extensions.