I personally am thinking that developing more programming languages has stopped making sense a long time ago because all of them ignore this untapped market that would help more languages.
C based FFI uniquely privileges C the same way the dollar being a global reserve currency privileges the US, but in the case of currencies the differences in how they work are very small, but with programming languages the differences are substantial.
Half the people working on new programming languages should have been working on GObject competitors instead.
One of the things that is difficult to grasp is that the best FFI is not native to a specific programming language, meaning it does not privilege any specific programming language. Of course this doesn't mean that it cannot be heavily inspired by a particular programming language, it just means that the implementation must be common to all programming languages.
What I mean by the latter is that the features need to be implementable in all programming languages and the easiest way to do it is to define common data types.
The big fallacy that the C FFI advocates present is that it is somehow fine to require full C compiler infrastructure and the C standard library to parse C headers and produce the ABI, but somehow it is the height of evil to define a fat pointer as a C struct consisting of a data pointer and a function pointer and mandating every language to use it, because it is not a C native datatype and only the mythical programming language C is allowed to be used for FFI. I'm pretty sure half the battle could have been won by C having defined common datatypes for the sole purpose of ffi in its standard library.
There have been enormous strides here. We have developed WebAssembly and WIT, with Extism as a chrome over then providing tower of babel between most programming language pairs. We have GraalVM and Truffle to unify languages under one compiler. We have LLVM bitcode to combine languages targeting native machine code for many architectures directly. Swift, Zig, Jai all implement C interop on their own terms. AI agents allow us to re-target algorithms between languages with no compiler interoperability. There are countless pretty darn good ways for programs to communicate through message passing - MachO XPC, Dbus, COM, gRPC, HTTP3 (+ , ZeroMQ, Fuchsia Zircon+FIDL, unix domain sockets. There are more programmers now than at any time in history.
Yes these can be used as alternatives sometimes, but they are not really the same thing. We need a streamlined way for reusable libraries (written in different languages) to be consumed by different languages while not being limited to a specific runtime or platform. When code from different languages needs to be able to run inside the same process, with low calling overhead and avoiding copying when possible.
Your answer just throws in anything that enables programs to communicate somehow, ignoring all the differences and tradeoffs to what is being discussed here. Many of your solutions lock you in to a specific platform, a language or add a non-trivial overhead like message serialization or add an unnecessary complexity to a program. Also, IR does not solve the same problem as ABI.
im just pointing out (a) a ton of work has gone into this, (b) yielding many options for different use-cases, (c) without dependency on c-abi/c-header.
I personally am thinking that developing more programming languages has stopped making sense a long time ago because all of them ignore this untapped market that would help more languages.
C based FFI uniquely privileges C the same way the dollar being a global reserve currency privileges the US, but in the case of currencies the differences in how they work are very small, but with programming languages the differences are substantial.
Half the people working on new programming languages should have been working on GObject competitors instead.
One of the things that is difficult to grasp is that the best FFI is not native to a specific programming language, meaning it does not privilege any specific programming language. Of course this doesn't mean that it cannot be heavily inspired by a particular programming language, it just means that the implementation must be common to all programming languages.
What I mean by the latter is that the features need to be implementable in all programming languages and the easiest way to do it is to define common data types.
The big fallacy that the C FFI advocates present is that it is somehow fine to require full C compiler infrastructure and the C standard library to parse C headers and produce the ABI, but somehow it is the height of evil to define a fat pointer as a C struct consisting of a data pointer and a function pointer and mandating every language to use it, because it is not a C native datatype and only the mythical programming language C is allowed to be used for FFI. I'm pretty sure half the battle could have been won by C having defined common datatypes for the sole purpose of ffi in its standard library.