[jerf]

This conference is more likely to take you farther away from that goal than get closer. The foundational reason [1] that components are not interoperable is that they function in different guarantee regimes. A pure language can not simply "interoperate" with code that expects to mutate. Yes, the pure language can wrap around the mutation-based code, but it will forever have an impedance mismatch.

You can't trivially interoperate with languages that have different async/sync "colors", per the recent discussion: https://news.ycombinator.com/item?id=8984648 . And the discussion is good, because everything else that people mention as other things they thought the post was getting at are also examples of places where the code can't just trivially interop.

And I say this will likely take you farther away than closer because right now research tends to be focused on how to use colors more effectively, and how to use more colors in code rather than less. All this mitigates pretty strongly against cross-language interoperability. At least, against interoperability beyond what we have now, which I assume that you are not satisfied by.

A lot of code today is "interoperable" because a lot of languages inherit the drab gray of C. For all the apparent differences between Python, Perl, Ruby, Lua, and C, in the end they are all languages of a very similar color. Python, Perl, Ruby, and Lua amount to programming in C where every line is a complicated function call instead of a simple C statement, but you're still fundamentally in C. This has provided an illusion that interoperability is either somehow "easy" or something that could be improved, but as we move away from C's drab gray, interoperability is going to get worse, not better.

Perhaps someday if we settle on a particular rainbow, some form of code that can be sufficiently accurately labeled with its colors could be interoperable, but we're going to farther from that goal before we get closer.

[1]: I say this is the "foundational reason" because it's the fundamental one that can't be fixed by "just trying harder". There are also incidental reasons that often depend on exactly where in the lifecycle of the language C-interop was considered (compare Lua with Python, for instance), and accidents of history that persist because there's no point fixing them because the fundamental problem will remain.

[rectang]

    > right now research tends to be focused on how to use
    > colors more effectively, and how to use more colors in
    > code rather than less.

I understand that deprioritizing interop suits academics because new "colors" are more vivid when they don't have to mix with old ones, and also that commercial vendors have an interest in platform lock-in. But as a working programmer, I'm not satisfied by that.

Microservices are an interesting development in this space, providing an architecture within which components with very different underlying semantics may cooperate -- it's not a new idea, but neither was AJAX. Other historically significant interoperability initiatives are CORBA, COM, SWIG and GObject.

Then there are the virtual machines which run multiple programming languages (JVM, CLR, Parrot) -- but it is very hard for a VM to provide a superset of semantics in order to support every last "color" for every supported language.

The project I work on, Apache Clownfish (a "symbiotic object system"), takes the opposite approach: instead of aspiring to provide a superset of semantic functionality, it provides a basic subset plus glue, and instead of aspiring to serve as a platform it is distributed as a library.

Working through issues like how to design an object system which is compatible with diverse GC/allocation schemes is fun and mind-expanding, and I'm looking forward to publishing some white papers. But I wish there was more existing research. If the state of interop were to advance, maybe it wouldn't be so expensive to migrate to new programming languages and the industry could evolve faster.

[jerf]

"I understand that deprioritizing interop suits academics because new "colors" are more vivid when they don't have to mix with old ones,"

It's worse than that... some "colors" basically break if you allow admixture. For instance, immutable and lazy languages come crashing down around your ears if even so much as one thing unexpectedly mutates, because the compiler rewrote the order so strongly that even if you think you can second-guess the evaluation order, you're wrong. It's why "unsafePerformIO" in Haskell is even more dangerous than you might think it is as a novice... it isn't just that it violates the ideals of Haskell, it's really unsafe! It's not an idle comment, it's a serious warning. It is possible to use it correctly but it's very subtle.

"Microservices are an interesting development in this space, providing an architecture within which components with very different underlying semantics may cooperate -- it's not a new idea, but neither was AJAX. Other historically significant interoperability initiatives are CORBA, COM, SWIG and GObject."

This is why I qualified my post by assuming you were talking about more than these things, because they exist and aren't going anywhere. Fundamentally you can always send a message from a runtime and get a message back in return, but if there's any particular problem with that, it's that it's an oversolved problem rather than a not-solved-enough problem; too many choices! But that is not the fault or responsibility of any one language.

"The project I work on, Apache Clownfish (a "symbiotic object system")... Working through issues like how to design an object system which is compatible with diverse GC/allocation schemes is fun and mind-expanding,"

And I'd point out that a quick glance at your project (which I specify so you can correct me if I'm wrong) makes it appear that I actually already called out all the languages that currently supports (C, Perl, Ruby) [1] as working in the same fundamental regime, before I knew about what you were talking about. It sounds like you're working on those accidental complexities, which may well bring great value to many people (and let me highlight that, I'm serious, this may well be an awesome project, let nothing I'm saying here be construed as a criticism of the project, its ideas, or its worthiness of being worked on!), but is currently working on C-gray to C-gray interop, which, alas, does little to counter the ideas I'm discussing here.

If you are interested in these ideas, you may find it worth your time to dig into Haskell/GHC a bit and just consider how you might even begin porting your ideas across. By no means would I suggest actually putting any time to code this in, but it might give you a different perspective to think about. If you are feeling less ambitious, even just looking at how you'd port it to Go, a language still mostly C-gray but with one colorful streak of asynch in it, could be an interesting compare & contrast.

[1]: https://github.com/apache/lucy-clownfish/tree/master/runtime

[skybrian]

I think this will be solved (well, managed) by using unrestricted "colors" as glue languages at the top of the stack and restrictive colors for portable, embedded code that should be sandboxed. Pure functions shouldn't be hard to call or to cross-compile.

[actsasbuffoon]

I'm not sure about this. For instance, Scala has a very capable Java FFI, but using it can be problematic due to the different expectations of the languages.

In Java, you represent a missing value with null, but well-typed Scala should never encounter a null. Java will also regularly raise exceptions, but Scala usually prefers wrapping values that may fail with Either. There's no clear way to automatically bridge the gap, which means falling back to Java is always going to be a necessary evil, rather than something that should be encouraged.

[skybrian]

Scala isn't really designed for embedding but something else might work. Perhaps a little language specifically designed for embedding. Some existing little languages include regexps and SQL, but perhaps there's room for portable pure functions.

But you do have to decide how data gets passed in and out.