[logicchains]

I'm really impressed by how backwards compatible it is. I just changed the Clojure version from "1.6.0" to "1.7.0" for one of my side projects, without updating any library versions, ran it, all the tests passed, and it seems to work perfectly. It also didn't break my Emacs setup, which is a breath of fresh air compared to how much work it was to get the Haskell tooling working with the new 7.10 GHC release (GHC-mod for instance still doesn't have a compatible release, although the trunk mostly works with 7.10). Similarly, even though it's months after the 7.10 release there are still libraries that don't support it, like reactive-banana-wx, whilst a couple of the Clojure libraries I'm using haven't been updated in over a year yet still work fine on 1.7, and none of the libraries I'm using break on 1.7.

To be fair, GHC and the Haskell ecosystem is far more complex than Clojure and its ecosystem/standard library. Nevertheless it's pleasant how easy Clojure was to upgrade (although of course this stability is nothing special: more conservative languages like Go and Java generally break almost nothing on upgrade).

[kul_]

Yes this is something clojure users take for granted. I dont know the state of Scala right now but a year back even minor version bump was horrible in scala in terms of backward compatibility. Really impressive job by clojure code devs in terms of maintaining such stable releases.

[bad_user]

Scala's minor versions don't break compatibility. A year ago Scala 2.11 (major) was being released, which means you're talking about either 2.10.x or 2.11.x, neither of which broke binary compatibility between minor versions. And major versions usually have source level compatibility, our upgrade from 2.10 to 2.11 being pretty smooth.

The fundamental difference is that Clojure gets distributed as source and not as compiled .class files. Being an interpreted language that gets compiled on the fly does have some advantages. But it also has drawbacks. Startup performance suffers, the Java interoperability story is worse, many tools such as Android's toolchain expect bytecode, etc ...

The problem that Scala has (and also Clojure, as soon as you do AOT) is that Scala's idioms do not translate well into Java bytecode, as Java's bytecode is designed for, well, Java. Therefore even small and source-compatible changes, like adding another parameter with a default value to a method or like adding a method to a trait, can trigger binary incompatibilities.

The plan for Scala is to embed in the .class, besides the bytecode, the abstract syntax tree built by the compiler and then the compiler can take a JAR and repurpose it for whatever platform you want. This is part of the TASTY and the Scala-meta projects. If you think about it it's not that far from what Clojure is doing, except that this is done in the context of a static language that doesn't rely on the presence of an interpreter at runtime. Of course, LISPs are pretty cool. And of course, you'll still need to recompile your projects, but at least then the dependencies won't have to be changed.

[puredanger]

Clojure itself actually is distributed as compiled .class files. We take some effort to ensure that there are not changes that break binary compatibility from AOT-compiled Clojure code (which is not uncommon) and break its calls into the Clojure compiler or runtime.

[pron]

> and also Clojure, as soon as you do AOT

Not so much. You can do separate compilation with Clojure to a far greater extent than Scala (perhaps even completely). The Java interfaces of Clojure functions haven't changed in incompatible ways in a very, very long time.

[bad_user]

The Java interfaces that Clojure relies on don't matter that much. On the other hand what is the bytecode representation of a protocol or of a multi-method?

Scala has an equivalent representation for everything it has. For example traits are just Java interfaces but with corresponding static methods. Another example is default parameters (a concept Java doesn't have) is done with method overloading. The Scala compiler has to infer Scala-specific stuff (like signatures using Scala features) from compiled bytecode. Functions that aren't methods (e.g. anonymous functions) get compiled either to static methods, or to classes that have to be instantiated (in case it's about closures closing over their context). Scala does not have the concept of static methods, but it has singleton objects, which can also implement interfaces. Of course, these get compiled to static methods, but there's also a singleton instance instantiated for those cases in which you want to use that singleton object as a real polymorphic instance. Etc, etc... So there's a protocol in place for how to encode this in the bytecode, there's a protocol for everything. And so even small changes in Scala's standard libraries can trigger big bytecode changes that end up being backwards incompatible.

Clojure doesn't have to do this, because Clojure dependencies get distributed as source-code, as I've said.

[pron]

> Clojure doesn't have to do this, because Clojure dependencies get distributed as source-code

Clojure doesn't have to do this because it was designed to allow separate compilation as much as possible[1], and because it hardly ever changes binary representation in backwards-incompatible ways. Protocols and multimethods are indeed handled at the call-site, but in such a way that a change to the protocol/multimethods don't require re-compilation of the callsite[2]. Similarly, Kotlin, a statically typed language distributed in binary, is also designed to allow separate compilation as much as possible[3]. If a feature would break that property (e.g. require re-compilation of a callsite when an implementation changes at the call target), that feature simply isn't added to the language.

This is a design that admits that extra-linguistic features (like separate compilation) are as important as linguistic abstractions (sometimes more important).

BTW, separate compilation isn't only a concern with Java class files. Object file linking also places limits on how languages can implement abstractions yet still support separate compilation. Some languages place less emphasis on this than others.

[1]: In fact, as a Lisp, Clojure's unit of (separate) compilation isn't even the file but the top-level expression. No top-level expression should require re-compilation if anything else changes.

[2]: The implementation of protocols: https://github.com/clojure/clojure/blob/master/src/clj/cloju...

[3]: Even Java doesn't support 100% separate compilation. There are rare cases where changes to one class requires recompilation of another.

[bad_user]

In my opinion choosing a different language than the host (in this instance Java) has to bring enough advantages to balance out the disadvantages, like less (idiomatic) libraries, less documentation, less tools, less developers available on the market. Not to pick on Kotlin, I'm sure there are people that love it, however I personally don't see what advantages a language like Kotlin brings over Java 8, being CoffeeScript versus Javascript all over again. With Clojure or Scala we are talking about languages going into territories that Java will never venture into, for the simple reason that Java is designed to be mainstream, which really means appealing to the common denominator. Of course, as we've seen with CoffeeScript, the market can be irrational, but it started to go away already, after a new version of Javascript plus the realization that everything brought by those small syntactic differences was bullshit.

Going back to Clojure and the need or lack thereof to recompile call-sites, given that Clojure is a dynamic language that doesn't have to concern itself with much static information and that does get distributed as source-code, I feel that this is an apples versus oranges discussion. But anyway, lets get back to protocols. So protocols do generate corresponding Java interfaces, just like Scala's traits. Except that Clojure being a dynamic language, there isn't much to do when your functions look like this to the JVM: https://github.com/clojure/clojure/blob/master/src/jvm/cloju...

There's also the issue that Clojure's standard library has been more stable. Well, that can be a virtue and in the eye of the beholder can be seen as a good design, however it has many things that need to be cleaned out. As a Clojure newbie I couldn't understand for example why I can't implement things that work with map or filter or mapcat, or things that can be counted, or compared, only to find out that its protocols and multi-methods aren't used in its collections and that there isn't a Clojure specific thing I can implement to make my own things that behave like the builtins. It's also disheartening to see that the sorted-set wants things that implement Java's Comparable. Clojure's collections have sometimes surprising behavior - like for example I might choose a Vector because it has certain properties, but if you're not careful with the operations applied you can end up with a sequence or a list and then you have to back-trace your steps (e.g. conj is cool, but the concept should have been applied to other operators as well IMHO). Transducers are freaking cool, however I feel that the protocol of communication isn't generic enough, as I can't see how it can be applied to reactive streams (e.g. Rx) when back-pressure is involved (might be wrong, haven't reasoned about it enough). As any other language or standard library, Clojure is not immune to mistakes and personally I prefer languages that fix their mistakes (which I'm sure Clojure will do).

EDIT: on "separate compilation", not sure why we're having this conversation, but generally speaking Scala provides backwards compatibility for everything that matches Java. Adding a new method to a class? Changing the implementation of a function? No problem. On the other hand certain features, like traits providing method implementations or default parameters are landmines. Along with Java 8 things might improve, as the class format is now more capable. As I said, there's a roadmap to deal with this and with targeting different platforms (e.g. Java 8 versus Javascript versus LLVM) through TASTY, but I don't know when they'll deliver.

[mateuszf]

That's just the power of good upfront design.

[sreque]

This doesn't seem true. See http://comments.gmane.org/gmane.comp.java.clojure.user/85930

"Binary compatibility across releases is not guaranteed, but it is something we strive to maintain if possible. "

[pron]

If you read what it says, they've changed an implementation of a function which may change ordering of unordered collections, and that may break some people's code if they relied on some specific ordering. They are not talking about breaking interfaces. In fact, they specifically say they are not aware of any binary incompatibilities. Their "strive to maintain" works out very well in practice, so far.

[puredanger]

I wrote that sentence and you should take it as written. In general, we strive to maintain binary compatibility (old AOT-compiled Clojure code should continue to run on newer version), we do not exhaustively test for or guarantee that. At times we have broken this in alphas etc and we take it as a high priority to fix such things.

[xixixao]

Is the job simpler because Clojure is not statically typed?

[arohner]

I'm not a Scala user, but Clojure the language tends to be extremely stable. Existing stdlib functions almost never get updated, unless they add a new (backwards-compatible) arities.

Releases tend to consist mostly of new features, and a small number of bugfixes.

[yogthos]

Part of it is that the standard library API has been stable for years, where new things are added but no breaking changes are introduced.

The second reason is that Clojure libraries are typically shipped as source. This avoids the whole binary compatibility problem you see in Scala. Typically, you only compile the end applications to byte code.

[pron]

I don't think so. It's because of a strong commitment to users not to introduce breaking changes. I've been mostly using Java and Clojure in recent years, and before that C/C++ and I'm pretty surprised people don't take backwards compatibility for granted, and that there are languages that don't consider it a top priority. (Well, new C++ compiler versions sometimes broke existing code, but that was quite rare)

[dragonwriter]

Probably, more because its a lisp, whereas Scala was more of a fundamentally new language that isn't a member of such a well-studied family, but instead was led by inspiration from lots of different places, and has a lot more work in shaking out how to reconcile those influences into a coherent whole.

Though Scala being not merely statically typed by aimed at both interoperating with Java's type system and supporting a more robust and powerful type system than Java does did pose particular challenges.

[bad_user]

No, it's because Scala's dependencies / libraries get distributed as compiled .class files, whereas Clojure's get distributed as source-code. Tried explaining this above: https://news.ycombinator.com/item?id=9807323

[raspasov]

Or because it's a lisp? : )