Tell HN: Rust is the superglue

[itsmefaz]

I am going through an existential crisis as a developer. I am looking to revamp my entire skillset to align more with where I believe the future of software engineering is heading.

And during this phase, I encountered Rust and loved it, but I couldn't wrap my head around its existence. Rust has a good balance of low-level and high-level abstractions, but I didn't know where to use them. For web and mobile, there is the JS ecosystem. For machine learning, we have the Py ecosystem. I see members of the Rust community thinking of Rust as a replacement for these languages, instead the right approach is to focus on enhancing these languages and becoming the backbone for this two ecosystems. Rust is the superglue we needed!

[constantcrying]

Rust seems monumentally unsuited as a glue language like Perl or Bash. It is statically typed, compiled, makes strong assumption about the code, is relatively hard to write/learn and somewhat verbose.

Rusts reason for existence and main goal is replacing C++.

>For machine learning, we have the Py ecosystem

A very bad state of affairs to be sure and also not really true as python is just a thin layer ontop of the actual code running the networks, which are written in C++ or loaded of to GPUs.

[bjornasm]

>python is just a thin layer ontop of the actual code running the networks, which are written in C++ or loaded of to GPUs.

Or Rust which is under libraries such as Polars. In other words: python is the glue in the data science world.

[nateb2022]

Indeed. Python is the high-level glue that connects the low-level stuff. When you import tensorflow, for instance, you deal with simplified abstractions and don't have to think about all the low-level optimizations that are taken care of by C/C++/Rust.

Rust is not the glue. It is a building block.

[fxtentacle]

Just FYI, the market for C++ is excellent and has been for years, but I've yet to receive any well paid Rust project offer. My impression is that most people working with Rust are idealists (and not businesspeople) and the salaries reflect that.

[smabie]

Rust is pretty popular among crypto trading firms and those tend to pay very well.

[nateb2022]

Correct me if I'm wrong, but don't most crypto trading firms (which aren't ponzi schemes in some form) pay well because they are usually short (<10 year) lived enterprises that receive a lot of initial hype before receding into the background after rewarding the entrepreneurs with above-average compensation plans? As far as I know, most such companies fail shortly after they run out of venture capital

[jmspring]

C++ consulting or actual full time jobs?

[sacnoradhq]

You maybe prematurely optimizing. The technological singularity will be realized when algorithms develop new programming languages, new hardware description languages, and so forth. We're not quite there yet.

There is no universal answer or panacea tool. Evaluating which tool to use from the toolbox depends on a particular use and who else will be working with it. Being flexible and experimenting should, at a minimum, improve one's engineering instincts even if a particular approach fails.

There won't be many engineering jobs left once self-programming systems and AI become consolidated by megacorps. The remaining jobs will be the "janitors" and AI architects. We won't need or be able to use 27 million software developers, perhaps only 10-100k in 20 years.

PS: In the meantime, run a model that can maintain and refactor COBOL. ;) Or, specialize somewhat to what's hot right now without worrying too much about the distant future.

[jononor]

Basically all companies over a certain size will need software development capabilities, just to improve their internal processes. There are over 350'000 companies with over 250 employees. If they have just 3 full-time equivalents each, that is still 1M jobs. With increasing degree of automation in companies in general, the potential value created from software increases. Many industries are still in the start of their digital transformation, even if they started over 10 years ago. Even in modern economies, many business processes are largely ran based primarily on humans with pen and paper. Even with vastly better tooling, I do not think the job market for developer will go down in the next 20 years. We will just create more software, tackle more problems. And the role of the human will continuously evolve along with our tools.

[fxtentacle]

Another niche: Many banks in Europe still use M4, but the existing maintainers are growing old and retiring.

[jzwinck]

What do they use m4 for?

[benji-york]

m4 the macro language?

[fxtentacle]

Yes. I heard about it being used for email automation and reporting.

Unrelated talk about the M4 language that I'm referring to: https://m.youtube.com/watch?v=ULZxHSPWn98

[FrustratedMonky]

First. I love F#. But it didn't take off in the market, so I'm learning Rust.

What I tend to see is that Rust does have a higher learning curve, because you have to change your brain's way of approaching problems. The 'functional' way of viewing things. (plenty of past flame wars on functional programming, so not trying to start that here)

Once you do learn Rust (or functional), it is safer, better, faster. But if you just came in from C++, Java, C#, and try and force that way of thinking into it, then it is difficult and crappy and you wonder what the fuss is about.

For Python. I tend to think if it wasn't for Machine Learning Libraries, it would have started to die off. It is super great as an ad-hoc, throw together some janky little script to tie some libraries together. It is very useful for new or non-programmers to just whip out something 'close enough'. Or as another poster said, the REPL that is similar to MATLAB. BUT -> Nobody builds big applications in MATLAB. So the use case is either, small janky scripts to do something small and useful, or as MATLAB with REPL which are also small scripts. Once a program gets to a certain size, it would be much better to graduate to something with at least type checking.

I do really hope that the giant ecosystem of Python does start to migrate to RUST. Ecosystems don't turn on a dime, so it will be years. And who knows, Python is also improving, maybe it's shortcomings will be fixed faster than the ecosystem migrates to RUST.

[smabie]

Rust isn't really a functional language at all? It's pretty much just an imperative language with some type level features.

[sn9]

Rust has strong functional influences [0] to the point of often being described as an ML with a C-like syntax and manual memory management instead of GC.

Like Haskell [1], writing in a higher level style can give the compiler enough information to get you non-trivial performance gains [2].

It's not just an imperative language. It's multi-paradigm and can be written in a more functional style.

You can just look at the ML family of languages and see which features Rust has that C/Java/Python don't.

[0] https://doc.rust-lang.org/book/ch13-00-functional-features.h...

[1] https://stackoverflow.com/questions/35027952/why-is-haskell-...

[2] https://ipthomas.com/blog/2023/07/n-times-faster-than-c-wher...

[ojosilva]

You can practice your Rust skills by writing performant and/or gluey extensions for higher-level language such as NodeJS (checkout napi-rs) and Python or complementing JS in the browser if you target Webassembly.

For instance, checkout Llama-node https://github.com/Atome-FE/llama-node for an involved Rust-based NodeJS extension. Python has PyO3, a Rust-Python extension toolset: https://github.com/PyO3/pyo3.

They can help you leverage your Rust for writing cool new stuff.

[ThePhysicist]

I think Python is probably way better suited to be the superglue you mention, and in many areas it already serves that function. Many Python libraries are high-level bindings for low-level C/C++/Fortran (or recently, Rust) libraries. Examples are numpy, scipy, tensorflow, PyQt, ....

Rust is great in the sense that it is very suitable for being embedded into other software as it doesn't have a runtime system (like e.g. Golang) and can be compiled for almost any architecture. Rust's usability for "regular" programmers and technical users is horrible in my opinion. I e.g. worked with scientists a lot (and was one myself) and in my lab everyone could wrap their head around Python in a couple of days and become productive very fast, whereas most people would throw in the towel when they had to write low-level code in C/C++. In terms of learning curve and complexity I would put Rust somewhere in the same ballpark with C++. There are many differences and Rust has a much better developer experience as it has a modern package manager and the toolchain is much nicer, but it's still a low-level language with many intricacies.

[the__alchemist]

Rust can be used in a number of domains; from the use cases of programming languages I've encountered, its niche isn't one of the things you mention, and it wouldn't be my first choice. I think Rust's strengths are in these domains:

  - Embedded devices
  - Standalone applications
  - Operating systems / RTOSes etc
  - Performance-critical libraries

[solomatov]

IMO, it's a replacement for C/C++ with higher safety guarantees. Of course it's more suitable for ML, and Web, than C/C++, but if there're no hard perf requirements, there're better options.

[metroholografix]

Rust is a welcome addition to the global language toolbox but I feel is mostly a mirage in terms of future breakthroughs in the language / programming space: It still falls under the "formal methods / program correctness" paradigm which has led people astray for decades. Like Gerald Sussman proclaiming (talking about Haskell):

"This is the most advanced of the obsolete languages!"

Watch this talk: https://www.infoq.com/presentations/We-Really-Dont-Know-How-...

Another great talk on the subject (also the biological model of dealing with complexity) is Alan Kay's "Programming and Scaling".

[quectophoton]

And still for all the focus on projecting safety and trust of the compiled programs, you'd think it would be relatively easy to compile `rustc` from source so that people and Linux distros don't need to trust the `rustc` from the GitHub repository.

But unfortunately you need to do a gazillion compilation steps if you want to build the latest `rustc` from scratch.

I honestly can't wait until gccrs is good enough to allow us take a shortcut and be able to bootstrap a `rustc` in a reasonable amount of time and steps.

[solarkraft]

The Element team is writing its next generation Matrix SDK in Rust, which means that it can be embedded in pretty much any language.

[nonameiguess]

Most of the early intention and excitement was the potential to replace C++, and maybe even at least a little C, in systems-level and native code. Obviously, it was first created to replace the JavaScript engine in Firefox, which was C++, but I know a lot of people working on basic utilities and libraries found all over the core components of any normal Linux distro who had hope we might get something equally performant and expressive to C++, but with a far better developer experience and language-level support for the more modern features that C++ tends to get from something like boost. Or look at things like ripgrep and what not, just trying to replace some of the common GNU coreutils, POSIX command-line things you expect a system to have. It eventually made it into the kernel, even, though only for device drivers and that's probably as far as it will get.

I would say this has so far not really panned out. Partly, this is because C++ remains and has always been good enough. Partly, because POSIX component replacements were not made as drop-in replacements, so you still need GNU coreutils and existing bash built-ins and what not if you want build scripts and system scripts to still work. Partly, because so much of the modern developer ecosystem is focused on web development rather than native system-level tooling and libraries that Rust has gravitated toward that because that is who is adopting it. Look at how, for instance, Rust versus Go has become so much more common of a question to see compared to Rust versus C++. It's clearly more expressive and full of features than Go, but unless your system really depends on very predictable latency to the point that garbage collection can't be tolerated, it's not clear there's any real value-add even if it's a "better" language.

Rust also suffers a bit from a problem faced by any new language on the scene. If you work in older languages like C, C++, Java, you automatically and easily interoperate with some many existing deployed systems. There's a tested, true, well-maintained library for just about anything you could ever want to do. You can start a project quite quickly by forking something similar that already exists. Languages that did well as up-and-comers had some compelling killer app they were connected to that gave them that ecosystem within some more narrow niche. JavaScript obviously runs in the browser and was the only option for a long-time if you want client-side code to execute in a user's browser. This also proved to be useful for cross-platform desktop apps when computers became powerful enough and disk space plentiful enough that just shipping a browser runtime with every application became a feasible idea. Go had Kubernetes and Docker's rewrite, so if you want to do anything related to container runtimes or orchestration, it's the obvious place to turn. Python came to dominate machine learning because it has great interoperability with native code and it could easily wrap pre-existing Fortran and C++ to get a BLAS and higher-level numerical libaries in a dynamically-typed language with a REPL that was more or less perfect for exploratory, interactive work, and the dreaded operator overloading that developers hate proved to be a great feature for attracting scientists, because it allowed SciPy and NumPy to mimic the syntax of MATLAB, giving a free version with near parity of something that normally costs five grand for a single-user license and researchers were already familiar with using.

Rust unfortunately sits in this unholy middle ground for easy adoption. It comes with a very nice build tool and package manager built-in, but the ecosystem unfortunately took on the JavaScript "bring in thousands of tiny libraries that each do one thing" characteristic that is antithetical to C and C++ with gigantic libraries that do everything. The syntax is like 80/20 ML/Algol, so people coming from a functional background will see it and wonder why it's so verbose and can't do better type inference and needs brace-delimiting. People coming from an imperative background will translate the classic "writing C in C++" thing to "writing C++ in Rust" because they don't fully grok the idioms.

The compile-time garbage collection thing is a great, very cool feature, not quite as good as something like Idris, but probably the best you'll get out of any language anyone actually uses, but it still sits in this weird middle ground. People coming from C and C++ will get frustrated feeling the compiler doesn't trust them but they're pretty sure they know what they're doing is correct. People coming from dynamically typed or garbage collected languages will just follow the suggestions coming from the rustc error messages until it finally compiles, but without really understanding what the hell the borrow checker is doing or why they need it.

If your idea is correct, then you're talking about replacing C++, which is the current backbone for JavaScript and Python. But think about what this entails. You're talking about rewriting the Chromium JavaScript engine on the one hand, and rewriting BLAS, LAPACK, Armadillo, TensorFlow on the other. Who is going to do this? You're talking about low-level systems programmers on the one hand and scientists, applied mathematicians, and engineers on the other. But most actual uptake for Rust is application developers working on relatively greenfield projects.

[burntsushi]

Maybe it hasn't "really panned out" because you're making up your own stories. For example:

> Or look at things like ripgrep and what not, just trying to replace some of the common GNU coreutils, POSIX command-line things you expect a system to have.

This is just not true. I never made ripgrep to replace grep. And you acting like it not being a drop-in replacement being a mistake suggests you're totally missing the point. See: https://github.com/BurntSushi/ripgrep/blob/master/FAQ.md#pos...

The goal isn't "C++ has to literally die." That's just dumb. It is not going to die in the lifetime of anyone alive today no matter what anyone does.

[andrewstuart]

Ripgrep puts the code search in my JetBrains ide to shame.

Ripgrep is so fast that it instantly returned results and I had to check it was actually doing anything.

I felt I wanted ripgrep built into a whole bunch of other things.

[bionhoward]

nah man, the reason python projects take longer than expected is usually the crappy dynamic type system. If you use Rust, then your tests run instantly (yes, instantly) and you can immediately see the issues in your code highlighted by e.g. rust-analyzer. In comparison, python you gotta run your code and wait. Yeah, you can use Nodemon to run it automatically but it’s still waiting around for bugs and crashes. Just doesn’t seem reliable. Once you reach “IntoIterator” level of Rust, it’s basically python except the type hints actually do something besides communicate to developers.

Further, Traits are 100% revolutionary over OOP because they allow you to separate the data structure from the functionality in a way more granular way than classes. Then you can have multiple implementations per structure with different input trait bounds so your data structure can react intelligently to various input data types.

Plus, match statements, they’re incredible, but you can’t use them in python unless you want to give up backwards compatibility. Rust has them and they work great.

As someone who used to write a lot of Python, and now writes a lot of Rust, I would meekly suggest the learning curve of rust is not so bad compared to all the downsides and drawbacks of Python.

JavaScript is a nice language too but has similar issues in terms of poor type system and TypeScript is basically the same thing as Python with type hints. They don’t use the type system fully.

If you have anything beyond a minor script, and even then, you could potentially have a better result with Rust over Python or JS (yes, even for scripting or websites!) over c and c++ idk because I don’t use those languages but rust feels quite pythonic to me and I doubt they have nearly the same readability

[Capricorn2481]

Typescript is way, way, way better than Pythons types. I would argue it's one of the best type systems and I've used a lot.

[ThunderSizzle]

The only bad part about typescript is that it still only transpire into JS. I wish they forked it to compile into MSIL or whatnot.

[0xfedbee]

This is not /r/rustjerk

[bionhoward]

Rust is awesome and could replace both JS and PY at exactly what they’re supposedly good at. The main issue is compatibility and library availability.

[charlieyu1]

Not a professional, just wrote some Python. I definitely find Rust difficult to learn. Mutability, variable borrowing, variable lifecycles etc, do I really need this? Personally I only want to learn another language rather than Python for performance issues, and C++ is far easier and a lot more intuitive.

[smabie]

Rust is a lot easier to learn than C++.

[sgu999]

I wouldn't be so sure for Python. It's getting cluttered by the new additions, but what it's very good at is making very concise programs that, while brittle, can achieve a lot. Rust is mostly the opposite of that... I've been coding almost exclusively in Rust for the past two years and love it, but I really don't get why we should have universal tools.

[andrewstuart]

Zig seems more interesting but it’s not mature enough yet.

I’ve been so spoilt with IDE support, documentation, massive community and ChatGPT for Python typescript and others that I really can’t pick up Zig yet.

One of the things I heard is that ChatGPT doesn’t actually “get” the hard parts of rust, and that’s a huge issue because the whole point of AI in programming is it’s meant to be able to lead and guide you in learning and understanding.

“Rust is so hard to learn that the AI doesn’t get it” is a bad place to be for rust.

[_jyty]

>"Rust is so hard to learn that the AI doesn’t get it" is a bad place to be for rust.

I don't think this is really a big point against Rust; in my experience, ChatGPT doesn't write good code in general.

To be honest, I think Rust's borrow checker won't seem very hard if you've used C++ smart pointers before, which are like Rust's references/Box<T>s, but there's no borrow checker. The borrow checker just formalizes and enforces existing good practice for using pointers in C and C++, and makes sure they're followed 100% of the time unless specifically opted out of.

What actually seems more interesting to me than Rust is the ATS language. Rather than a borrow checker, you have proof objects that you pass around with pointers (and these proof objects exist only at compile time), which prove to the compiler that the pointer has a valid address. What's cool is that you can do things like pointer arithmetic in a type safe way. The downside is that it's more verbose than either C or Rust. I'm still trying to learn it.