>Why do people keep thinking they can use a garbage-collected language for “systems programming”?
Why do you think it can't be done?
It was already used for systems-programming in the early 80s; whole machines were programmed in Lisp in low level: TI Explorer, Xerox workstations and others.
Why do people keep thinking that "systems programming" is a term with a well-defined meaning?
I've seen it used for anything from operating system kernels to database systems to anything that talks to the network to command-line utilities like ls.
Garbage-collected languages are fine for all of these except maybe kernels. But even there, you might use a system that lets you do critical sections in a mode where the GC is disabled, or in a sublanguage that is guaranteed to be GC-free.
Well, it's been happening since the '80s when several OSes were written in ostensibly garbage-collected languages. So it might be a little late to complain.
Because of lisp machines, Smalltalk/the Alto - the popularity of java and the popularity of golang? As well as a disagreement about what "systems programming" means?
But when you can do real-time video processing in an (at the time) young alternative implementation of python[1] - I'm not sure what we're arguing about anymore..?
I always wonder exactly what kind of "systems" these people are programming, clearly not any with real-time constraints. It almost seems like "systems programming" got co-opted to mean "not interpreted". At least that's what I noticed with Go's marketing.
There are a few HFT firms, most notably Virtu, that use Java. But my understanding from having interviewed people that worked there is that it's so convoluted to avoid GC pauses that you might as well be using C++.
> I always wonder exactly what kind of "systems" these people are programming, clearly not any with real-time constraints.
Well Niklaus Wirth for one was able to design an entire OS using a language (Oberon) that had garbage collection.
> There are a few HFT firms, most notably Virtu, that use Java.
The hedge fund where I worked used Java. We didn't have problems with latency, although admittedly we weren't doing the really low-latency stuff. There are patterns that you can use in Java that basically emulate manual memory management. In Java this is a little harder than in C# because there no value types, but java.nio basically lets you do whatever you want, so you can always allocate everything up-front. In fact, just for fun I did this myself in an audio setting. So basically I had an audio engine with a ring buffer to pass messages to the event loop, and with a little care I was able to ensure that there literally no garbage that made it past the first phase (i.e. locals). Those are essentially free, so basically the GC wasn't getting any pressure at all. It wasn't that hard to write, and with the excellent profiling tools available on the JVM it's easy to see which if you're making use of longer-term GC sweeps or not.
Finally, I would add that there are scenarios where C++ moves into automatic memory management. std::shared_ptr is an example of (shitty) automatic memory management, but there have been efforts, notably by Herb Sutter, to provide precise GC-collection as a library. For some non-blocking multithreaded algorithms, GC schemes are actually necessary since allocation becomes one of the prime vectors through which blocking occurs.
So conclusion, while it's certainly the case that most systems programming is done in languages with manual memory management, it's a little less binary than you suggest. That said: I'm writing a DAW (zenaud.io) and for that I am using C++, mainly for memory management :)
Many operating systems that use manual memory allocation and/or reference counting techniques are nevertheless unsuitable for real time.
Modern desktop and mobile device operating systems often exhibit embarassing lulls in responsiveness that resemble pauses in a rudimentary garbage collector.
Real-time operation can be "bolted on" to a non-real time operating system as a small specialized kernel which has higher priority access to the CPU and its own, separate resource management.
I don't tend to think of hard realtime as "systems programming". It's related in that a lot of the good languages for one are good for the other, but they don't seem like the same fundamental problem space - note that almost no OS kernels and even fewer base userspace layers (libc or equivalent, coreutils or equivalent, etc.) are hard realtime.
> But my understanding from having interviewed people that worked there is that it's so convoluted to avoid GC pauses that you might as well be using C++.
I've done a very minimal amount of this... the gist is that you avoid GC pauses by avoiding allocation. This translates into reusing objects using pools, etc.... and the assorted complexities that come from having to explicitly manage object lifecycles. In critical applications you often want to avoid dynamic memory in C/C++, so it may not be all that different.
It seems to md that systems (with a 's') programming was a term lntroduced by Go, and I always understood it has something with a wider scope than system programming - actually "intermediate level" applications for which a GC is acceptable both time-wise and space-wise.
Yet for system (without 's') programmers (kernel, driver or embedded systems devs for instance) a GC is still a no-go.
Why do you think it can't be done?
It was already used for systems-programming in the early 80s; whole machines were programmed in Lisp in low level: TI Explorer, Xerox workstations and others.