Hacker News new | ask | show | jobs
by jfbaro 1693 days ago
Hey people, first, congrats for the great work on GCC for RUST, second I have a basic question here from someone who is not experienced with low level languages. Would it be possible (and beneficial to the community) to have a "compiler as a service" in the cloud (either GCC or LLVM based) that would have the most powerful hardware setup available to compile RUST? Really cheap/free per seconds of compilation... so anyone would be able to compile Rust faster and also ANY improvement would de added to this Service... and once new and more powerful hardware is available it could be shared and used by the community. I know we still have to work on improving the compilation times, but maybe having a shared compilation pipeline that can be used by everyone could somehow alleviate the pain a little. Thanks
5 comments

There is sccache (https://github.com/mozilla/sccache) so a first step would be looking to see why it isn't used more to see how to lower that barrier.

Another idea is crate-build caching so local and CI can pull down a pre-built dependency, rather than building locally. This would need to handle rust versions, feature flags, architectures, compiler settings, etc. This would most help CI since the result would get cached locally

The last idea I'm aware of in this area is watt (https://github.com/dtolnay/watt). If the design and implementation was finished to allow proc-macros (and maybe `build.rs` scripts) to opt-in to a sandboxed wasm environment, we could have a local and networked binary cache for these which would dramatically improve Rust build times (and security). Some people outright avoid proc-macros because of the build-time impact.

Distributed compilation isn't as helpful as it might seem at first glance. In order to compile something remotely, you need to slurp all the source files and the relevant environment information locally, send it to a remote compilation process, wait for it to compile, and get the results back. This extra network overhead is going to eat up a lot of the potential speed benefits you might get on a beefier machine, and limited bandwidth is likely to limit the maximum width of the parallelism you might encounter.

A better alternative than distributed compilation is artifact caching. Effectively, instead of shipping .rs files all over the place to compile them, just ship .o files around. Rust already has a decently canonical solution for this in the form of sccache, which was developed by Mozilla for caching the builds of Firefox using Amazon S3 as the storage (hence the name--S3 ccache [compiler cache]).

That compiler service is usually put together as part of a Continuous Integration process where if hosted in the cloud you can launch beefy instances on demand and or spot (aws jargon) as part of your development workflow/loop.
I don't mind my laptop not screaming obscenities at me every time I do a cargo build/run. The local changes will have to be synced frequently enough for this to work. May be have the remote compiler named "cargor" and let is seamlessly run the compilation step in cloud.

However this might not really work well at all unless the network connection is insanely fast as the build sizes are huge with rust (especially in debug modes).

That sounds terrible.
People who disagree with it. Why exactly?
Security - who knows what ended up in the binaries ?