[rao-v]

It distresses me that we don’t have a language that can do a best effort parallelization of arbitrary loop like code across SIMD, multiple threads, multiple cores and GPU with a small directive.

I don’t need it to be optimal, just … handy as an option!

The last time I brought this up here, folks offered a bunch of options that don’t quite do this, and the best candidate was this 15 year old compiler project that is Intel specific!

https://ispc.github.io/

Could some programming language nerd build this?

(While you are at it give me a clear idiomatic way to pay the cost to switch from array of structs to struct of arrays)

[saagarjha]

The problem is you need both a PL nerd and a performance nerd and while that group has some overlap so these people are not as uncommon as you’d think the task is pretty hard so you need a lot of people on it, with a bunch of funding, etc. Usually it’s just cheaper to rewrite all your code by that point and so these efforts fail

[rao-v]

It seems like such a tempting gap though. The sort of thing you’d think in 2015 would be an obvious capability of 2026 languages!

[inigyou]

The 2015 people didn't take into account the high stress levels in 2026. We're all struggling to pay for groceries and rent.

[brabel]

I think I am missing the joke here. Who exactly is worse off than 11 years ago?

[inigyou]

Pretty much everyone except the ultra wealthy.

It was bad 11 years ago. It's worse now.

[t-3]

Eggs 11 years ago cost $0.89 (hell, TWO years ago!). Today they cost $5+.

[rao-v]

Food inflation has been moderately high but egg prices were a mix of weird short term events and umm price fixing (look up the recent case).

They are back to pre 2022 levels now! https://fred.stlouisfed.org/series/APU0000708111

[tbrownaw]

> best effort parallelization of arbitrary loop like code across SIMD, multiple threads, multiple cores and GPU with a small directive.

I doubt GPU is included by most runtimes yet, but for the rest of that have you tried SQL?

[pjmlp]

Good answer, too many folks miss out how powerful SQL actually is, and with stored procedures its compilation to native code can even cached across executions.

[Rendello]

And for an Struct-of-Arrays approach, DuckDB and other columnar databases can have nice advantages (I mentioned this in another comment [1]), including optimizations you wouldn't see in typical code (SoA or otherwise) like column-level compression [2].

The big problem with databases, in my opinion, is the horrible API friction between them and your code (not even SQL per se). It makes sense if you're calling out to a database server and transferring data, but for the small, intermediate values we see in code every day, the relational model is amazing and yet so painful to use within a given programming language.

I've been envying the C# people and their LINQ, and the Java people and their jOOQ, because I'm either making a half-assed database in my own code with structs, arrays, and hashmaps, or I'm constructing some SQL monstrosity, shoveling it out to SQLite or DuckDB through a library, and marshalling the types back and forth.

Why can't I just have everything I want all the time?

1. https://news.ycombinator.com/item?id=49016824

2. https://duckdb.org/2022/10/28/lightweight-compression

[rao-v]

A genuinely funny and good point!

[antonvs]

> parallelization of arbitrary loop like code across SIMD, multiple threads, multiple cores and GPU

The reason we don’t have this is that it’s a holy grail, an unsolved problem, for quite fundamental reasons.

The other comment about SQL hints at why: SQL is largely declarative and has complex semantics built into the language, which allows for analysis and optimization that go beyond what’s possible for lower-level, general purpose languages, especially imperative ones.

For code in those languages, even just determining whether “arbitrary loop like code” is parallelizable is undecidable in general.

The challenge is that as you make a language expressive enough to describe arbitrary algorithms, you also make it progressively harder for a compiler to infer safe and useful parallel execution automatically.

Another big issue is that the various forms of parallelism are only similar at a very high level. They have fundamentally different execution models and constraints. Translating arbitrary imperative code to handle that essentially involves first inferring the intent of the code, then rewriting the code, including how data structures are organized, to fit the target architecture. This is far more than what ordinary compilers do.

There are also a lot of choices involved. Parallelism isn’t always free, so you’d need to make sure that the costs don’t outweigh the benefits - and you’d need to do that for many different decisions, like whether to use threads or not. Now you’d have a compiler building cost models to try to not make dumb choices - and without actually restarting and comparing alternatives, it’ll make mistakes.

In many ways, you’d be better off using an LLM for this, because that’s the level of understanding you need to have a hope of getting a good result.

That all said, you can do much better with more constrained languages or frameworks. SQL is the most successful example of that. Java’s streams and Rust’s Rayon only target multicore CPUs, but similar approaches could be used to do more. (Although you still potentially run into issues with optimal data shape across paradigms.) Languages like APL, J, and Futhark are all relevant.

The other family of solutions to this are the frameworks like Apache Spark, Apache Beam, and the ML frameworks like Pytorch. The latter lets you describe (tensor) computations at a high level, leaving the framework free to figure out how to implement them - much like with SQL.

[scottapotamas]

It's computer vision focused and might have been suggested previously, but I think Halide is a pretty good/mature demonstration of one way to approach this - writing the algorithm and the execution descriptions as separate passes with access to auto-optimisers and GPU runtimes.

[pjmlp]

Chapel aims to do that, however they only focus on HPC as userbase.

[maximilianburke]

ISPC isn’t Intel-only: https://github.com/ispc/ispc

[rao-v]

Good to know!

[imtringued]

>It distresses me that we don’t have a language that can do a best effort parallelization of arbitrary loop like code across SIMD, multiple threads, multiple cores and GPU with a small directive.

It's called ParaSail. The next closest thing to ParaSail is ...

... literally just Rust.

Why? Because ParaSail has completely eliminated pointers, thereby preventing pointer aliasing. It can't be understated that pointer aliasing and alignment are the two biggest bottlenecks preventing autovectorization.

The people talking about better compilers, etc, just don't get it. It's not a compiler problem, it's a language semantics problem.

Pointer aliasing prevents parallelism full stop. If there is a single memory region and you perform a write to it, you have to assume that the write invalidates all data loaded from the pointers. If you make pointer aliasing illegal, then you have guaranteed that each pointer points to a distinct subset of the global memory, turning each pointer into a pointer to an isolated region. This means you have multiple regions you can write to in parallel. This is crucial, if you do not understand this you don't get parallel programming at all.

I mean think about it, this is the difference between having a four toilet bathroom with a single door or four doors.

The thing about alignment is not as easy to explain, but here is my attempt at it: If you allow the array to be misaligned at the front, then you have to run scalar code at the front. Same problem if you misalign at the end.

If you have a naked pointer and a for loop (think C), then the alignment problem alone precludes autovectorization without a complex scalar preamble. Autovectorization has to assume that the loop length could be anything, meaning it could be less than 8 elements to begin with. If the loop is always a multiple of 8 and always aligned, then the loop can be autovectorized even if the loop only does a single iteration.

Of course, after these critical blockages are gone you're still stuck with the problem of having to write branchless/non-diverging code.

[Rendello]

I'd never heard of ParaSail, so I found this high-level overview [1]:

> All of the objects declared in a given scope are associated with a storage region, essentially a local heap. As an object grows, all new storage for it is allocated out of this region. As an object shrinks, the old storage can be immediately released back to this region. When a scope is exited, the entire region is reclaimed. There is no need for asynchronous garbage collection, as garbage never accumulates. Objects may grow in a highly irregular fashion without losing their locality of reference.

> Note that pointers are still used behind the scenes in the ParaSail implementation, but eliminating them from the surface syntax and semantics eliminates the complexity associated with pointers.

This approach seems to come up in many contexts, where a pointer-based address (raw pointer, reference, slice, etc.) is abstracted into a higher-level address key, usually an index integer (essentially a higher-level virtual pointer). The implementation might reallocate under the surface, or manage chunks of data through some sort of paging where the underlying pointers don't change (I was musing about this here [2]).

I guess I'm thinking out loud here, but most dynamic languages (eg. Python) don't expose the pointers or care about invalidation of the addresses, they just happily reallocate. I've barely written parallelized code, is pointer aliasing really one of the biggest roadblocks? It seems like it can be abstracted away fairly easily, even in a pointer-exposing language.

1. https://www.adacore.com/uploads/papers/parasail-pointer-free...

2. https://news.ycombinator.com/item?id=49013285