Hacker News new | ask | show | jobs
by eru 66 days ago
I agree with the author: it's a shame that TVars aren't catching on in more languages. They are a great idea from the database world, that we could use in the rest of computing, too.
6 comments

The entire programming (or even computing) ecosystem suffers from this issue where very useful ideas don't always propagate across domains even though they just make a whole lot of sense. I'm not sure if it's because they truly wouldn't work out in practice, or if it's just a discovery/communication thing.

One thing that I think do affect things, is that language design discussions tend to be concentrated into their own communities based on the programming language itself, rather than one "programming language discussions" place where everyone can easier cross-pollinate ideas across languages. Luckily, there are some individuals who move between communities without effort, which does lead to a bit of ideas making it across, but it feels like we're missing out on so much evolution and ideas from various languages across the ecosystem.

> Luckily, there are some individuals who move between communities without effort, [...]

Oh, many of these travelers spend a lot of effort!

It's discovery and communication. Public education for adults is way under-appreciated in many many scopes.
The cross-fertilization of ideas across computer science domains is more limited than I think people assume. Databases are just one area that contains a lot of good ideas that never seem to leak into other parts of the software world.

Supercomputing is another domain that has deep insights into scalable systems that is famously so insular that ideas rarely cross over into mainstream scalable systems. My detour through supercomputing probably added as much to my database design knowledge as anything I actually did in databases.

I've had this thought myself too. Going off on a slight tangent: I think there's also loads of useful stuff in domains like either of these which maps amazingly well to AI agent system design, but there's such a huge discrepancy between the knowledge bases of the fields that no benefit ever really surfaces.

(Speaking from the perspective of someone who simultaneously loves high-performance compute and agentic AI haha)

The canonical industrial explanation “why not” is probably this 2010 piece from Joe Duffy @ Microsoft:

http://joeduffyblog.com/2010/01/03/a-brief-retrospective-on-...

I don’t think we read the same thing.

> Models can be pulled along other axes, however, such as whether memory locations must be tagged in order to be used in a transaction or not, etc. Haskell requires this tagging (via TVars) so that side-effects are evident in the type system as with any other kind of monad. We quickly settled on unbounded transactions.

Snip

> In hindsight, this was a critical decision that had far-reaching implications. And to be honest, I now frequently doubt that it was the right call. We had our hearts in the right places, and the entire industry was trekking down the same path at the same time (with the notable exception of Haskell)

So basically not that TM isn’t workable, but unbounded TM is likely a fool’s errand but Haskell’s is bounded TM that requires explicit annotation of memory that will participate in atomicity.

Having worked a bit on a hobby STM in C++ (spun out of a DB startup) I would have to agree. Fully transparent STM that depends on a "sufficiently smart compiler" for an imperative language with unrestricted side effects is hopeless. But I do think that a much humbler version of STM is feasible for C++ or Rust, requiring much more explicit cooperation from the programmer. I haven't worked on this for 3 years but hope to revisit it someday.
Haskell still needs TVar and it’s not an imperative language with unrestricted side effects. I think it’s bounded vs unbounded. Side effects make it more complicated perhaps but it sounds like even in a JIT language you could have done it.
It's possible (I've done it) in C++ but there are huge footguns that I'm still ambivalent about. I agree that the bounded/unbounded distinction is the key: data must explicitly be tagged as transactional. One of the benefits of bootstrapping an STM from an existing DB as I did is that this explicitness is already present.
> unbounded TM is likely a fool’s errand

It's the whole language, not just the TM code. Other languages have no way of opting out of the TM code, whereas Haskell does.

Well, in a sense other languages opt-in to TM code by interfacing with a relational database. Similar to how in Haskell you put TVar in front of the relevant bits to opt-in.

In Haskell it's just more economic and better integrated with the rest of the language and its typesystem.

I think it is pretty sad how there are so many modern programming languages coming out that fail to actually do something novel.

The primary advantage of a new programming language is that there is no legacy code to be compatible with. The software ergonomics space has been thoroughly explored, but restricting programs to subsets with useful properties is still an untapped field. It's seen that way because professional programmers are not used to dealing with restrictions to expressive freedom. New languages are supposed to increase freedom.

The vast majority of new programming languages would benefit from the main language being as restricted as possible and then require you to opt into the features selectively.

> The primary advantage of a new programming language is that there is no legacy code to be compatible with.

Depends a lot on what you are doing. Clojure and Scala were new languages, but they had a lot of legacy code to stay compatible with. C++ falls in a similar camp.

Intel, MSFT, IBM spent billions from about 2005-2015 trying to make this happen and failed miserably.

https://dl.acm.org/doi/10.1145/1400214.1400228

It is a big reason why I picked Scala3/Zio over Rust for my most recent project.
Well, what means to support, truly, TVars?

Is easy, or hard?

Demand a new paradigm at large, or is only a inconvenience in the few places is used?

Because if the answer is "turns the language into Haskell" then is a big NOPE!

Here is a C++ STM based on the Haskell STM API:

https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2015/n44...

Well, Haskell is a big language with lots of different aspects.

To nicely support TVars, it's good if your language can differentiate between pure code and code with side-effects. Haskell's type system is one way to get there; but eg something like Rust could probably also be coerced to do something appropriate.

Apart from that, you probably don't even need static typing to make it work well enough (though it probably helps). You definitely don't need laziness or Haskell's love of making up new operators or significant whitespace.