I understand the need for memory footprint in some situations, but what's the point of seeking performance for a software that mostly calls LLMs and waits?
Before I tried coding agents my guess would have been: none.
But seeing how slow claude code and copilot cli are and how much ram they use I'm flabbergasted. If you have long running sessions they can both take tens pf gigabytes of ram and feel quite sluggish.
huh. my evidence with codex hasn’t been so bad. and tbh why would i discourage anyone from coding. hack away mr hacker. your solution will either sink or swim
The appetite for Rust is the appetite for higher guardrails. Automatic memory management in safe Rust makes it less likely your app bloats even as its source balloons.
The people "writing" agents are not themselves experts in how to write performant code. Claude Code is so massive and ugly it can only be realistically maintained by continuing to throw LLMs at it. But that's not a replacement for good software design.
I've been playing with running Claude Code inside a Vagrant VM. I can't be certain it was getting OOM killed when I allowed the VM 4GB of RAM, but when I went to 16 it did seem to be more stable...
> I can't be certain it was getting OOM killed when I allowed the VM 4GB of RAM
Of it's actually getting OOMed (and not backing off by itself), I'm pretty sure that's logged in dmesg. Or earlyoom or systemd-oomd if userspace is in play and getting there first.
I see spreading Rust as an overall good thing, because it changes benchmark on how software should feel in terms of performance, stability, memory footprint.
So even if it doesn't create tangible advantage in a particular use case - its still good for the whole industry.
Edit: I lost the context that this is about building devtools where you can’t just throw more hardware at the problem. But perhaps my answer still explains the reality: anthropic builds Claude with Claude so Claude needs to be easy to build with Claude.
Easier to read for humans is easy to read for LLMs. A more expressive language will bring about fewer misunderstandings when you apply stochastic tools like LLMs.
Just be sure you don’t choose something heavier/slower that is not more expressive.
Ada? Other than c and c++ everything else benchmarks 2-4 times slower than rust for compute bound tasks, even after jit warmup. I'm up for ada though, especially with an llm where I don't have to type all that verbose syntax.
Since when is Rust a systems programming language? On their website it is defined as "general-purpose programming language".
>Approaching every problem as a systems programming problem is a massive waste of resources and intellect.
If someone took their time and decided to use Rust maybe their first couple projects won't be as efficient, but they will learn the language and how to use it in the best ways, they will contribute to the ecosystem, etc. Again, I'm looking at the big picture and not specific project case.
I haven't used Rust extensively but my feeling is, if you change the design (which inevitably happens in many early stage projects), the refactoring takes more time due to borrow-checker semantics. Although I am far from a representative sample and could well have been using it wrong
When you write Rust long enough you settle on certain architectures (message passing, event loops) that go well with the borrow checker, and don't end up thinking about it too much. Plus you can always throw an agent at the first set of errors from the refactor and let the compiler guide the annoying parts.
The biggest reason to pass on Go right now (if your software can tolerate a runtime) is the lack of algebraic data types when doing interesting domain modeling. It makes such a huge difference it’s worth tolerating the pain points of Rust (or Swift, or F#) just to have them.
You can tune java runtime in many ways, achieving impressive throughput/latency for your type of workload.
Next to none of them will get you nearly as good cold start times as of native app, if using free java.
There was GraalVM and its ecosystem which included Java Native Image - first thing I’d evaluate if thought about non-server side, performant Java application.
But it all had been sadly swept away by Oracle from free tier.
I use GraalVM and Native Image now and while the project --a small CLI tool-- is tiny (2kLOC with mainly AWS-SDK deps) the compile times are huge (~3 minutes), the OS-dependencies many (so much I use a build container to ease the burden of installing all) and the resulting binary is huge (~60MB).
But then it distributes as one binary and starts in milliseconds.
Rust would have been a better fit (cargo-and-done, smaller binary, quicker to compile); but I wanted to use Kotlin as we use in all other projects.
It hasn't been swept away by Oracle, far from it. It's development is just no longer coupled to the OpenJDK release cycle, which benefits both projects.
What's the latency difference between a long running process issuing a network call in Java vs rust? This is such a short time that it is completely overshadowed by noise (OS doing something else, what other software is running etc)
As for throughput: you have 1-2 requests going at a time, the next one waiting for the reply. What throughput are we talking about?
That's like speeding to the post office and expecting your letter to get to the recipient faster.
you seem to specifically aim at the current example, but mine wasn't
Anyways, consider how higher memory usage can affect the systems performance dramatically once the system needs to start swapping memory to disk signficantly
If you cannot write a simple Java agent without consuming so much RAM that your system is swapping then that really says more about the developer than anything.
Java is used in plenty of embedded systems and other memory constrained environments. Yes, it’s not going to perform well compared with Rust, but that doesn’t mean it’s an Electron-equivalent bloated clusterfuck of an ecosystem that’s going to eat all your system resources.
> That's like speeding to the post office and expecting your letter to get to the recipient faster.
I mean, the post office is not a magic box. Actual people will take your letter somewhere, sometimes batching sends. So running to the post office might actually get your letter in an earlier batch, same as ordering on amazon or your online supermarket in the morning or in the evening might change the delivery time.
For the most part, doing things right in the given language matters more than change of language. A lot of refactors in Rust (in the coding agent space) I see jump straight to Rust without considering what inefficiencies can be addressed before changing the language.
Having said that, I considered a Go/Rust rewrite of Dirac (https://github.com/dirac-run/dirac) for some modules to support cases when someone wants to run like 30 agents, but it quickly became obvious that, a) while the node event loop is a bottleneck, it is not the sole bottleneck and b) if you have a VSCode extension, you can't totally get rid of TypeScript, so it just becomes the case of bi-lingual project and the maintenance burden that comes with it
Rust is just another language. Sure it's cooler than some langs, to some ppl. Sure.
The author made the choice. Open sourced it (thanks!). So now we all enjoy more options. Saying author did so because "cool" does not sit well with me. It's feels like you get a no-strings attached gift of significant value and then going saying the giver gave it to be seen as cool.
Opencode can be surprisingly hard on the CPU (could be an issue when coding on battery or a weak remote VM), and uses a lot of RAM. A little competition is always welcome.
Even a simple coding agent TUI should work instantenously, which I sadly cannot say is true about typescript-based applications like Claude Code or Gemini.
After switching away from GNOME Terminal + Zsh to Ghostty + Nushell, I started to appreciate how instant everything feels. Why not make everything just as fast?
I have to say this is one of my favorite things about local Qwen and Qwen code, it seems a heck of a lot faster that Claude and feels better to work with.
Problem is it is nowhere near as smart, so what speed I get in conversation gets killed by iteration.
That's true, but it's not quite the same thing. The single binary you're referring to is the interpreter and source code packaged together (at least for TS/JS).
If you install too many of these "single binaries" then at some point you would be better off just having a single interpreter and using npm/pip.
By contrast the Rust binary only contains the machine code for this program and can be directly executed.
That's exactly the tradeoff I made with Barnum (https://barnum-circus.github.io/). It's just not important to optimize the performance of the rust side for the reason you stated. So instead, all focus goes into making it easy for an LLM to build a reliable pipeline (from which LLMs are invoked).
I recall back in the mid 2000s when i saw many "rewrite in rails" apps. Its just hype, and it will die out in a few years when something new comes out.
But seeing how slow claude code and copilot cli are and how much ram they use I'm flabbergasted. If you have long running sessions they can both take tens pf gigabytes of ram and feel quite sluggish.