[pron]

I think it's part of a general trend of language designers (and even language fans) not understanding what makes languages work in practice. For academic languages this may not matter, but if you actually want your language to be used, you need to pay attention. A recent study[1] has shown that "extrinsic" factors (performance, tooling, familiarity, libraries) matter a lot more than intrinsic factors (syntax, abstractions) when it comes to adopting programming languages. A language that understood this beautifully was Java[2] (and, I believe, Go, which is very much in the spirit of the Java philosophy), that analyzed existing "advanced" languages like Scheme, Self and Modula (and indeed some of the people involved with those languages were among Java's early designers) and realized that the features that give those languages most bang-for-the-buck weren't clever abstractions but extra-linguistic features such as GC, dynamic linking and JITs[3], that can be packaged in a crude, "blue-collar" language that's familiar (and familiarity is indeed among the top factors in choosing a language, while "correctness" -- as in more powerful type system -- and particular language features are among the last). James Gosling called it a wolf in sheep's clothing. A wolf in wolf's clothing may look more fierce, and the sheep's clothing may indeed limit its maneuverability a bit, but it can still be almost as wolfish, yet much more palatable.

OTOH, Kotlin took whatever it could from Ceylon (the nullability types, flow-sensitive typing) while keeping 100% seamless interoperability with Java as the top priority, so it may not be a revolution, but its adoption costs are virtually zero. It doesn't even have much of a runtime library; it was designed in such a way that all cool features could be applied to Java's standard library. The idea was that abstractions are great as long as they don't come at the expense of more important features, such as availability of libraries, tools etc (Kotlin even supports Java's annotation processors, so popular libraries that rely on compile-time verification/code-generation such as Dagger can work on Kotlin code).

Of course, I take this much further than you and believe, unlike you, that the contribution of advanced language-level abstractions is not so pronounced at all (or, at least, it hasn't been shown to be significant), and obviously I have a very different perspective on Scala than you, but at least we can agree on something :)

[1]: Paper: http://lmeyerov.github.io/projects/socioplt/papers/oopsla201... Accompanying talk: https://www.youtube.com/watch?v=v2ITaI4y7_0

[2]: https://youtu.be/Dq2WQuWVrgQ?t=14m12s about why Java was designed the way it was

[3]: Sun had experimented with a JIT for Self long before it was introduced to Java, and the plan was to make Java exploit JITs almost from the get-go.

[eropple]

I think Go is a disastrous misstep (at least in the system-software contexts where I have to deal with it--obviously, you can do whatever for app languages), but I feel like your points as far as Java/Kotlin go are really well-put. I am honestly surprised at how happy I've been using Kotlin as the baseline server language for my current project (the backend of a fairly intricately orchestrated SaaS). I was not prepared to like it as much as I have; it's still very, very Java, but it blunts the worst parts of it from a syntactic and, where it can, a semantic perspective. Like--List<T> is readonly, but retroactively applied to java.util.ArrayList, similar to IReadOnlyList<T> in .NET. It's pretty nice. I am a Scala person, or I'd like to be but I find I write unmaintainable junk in it; Kotlin provides many of (though not all; I desperately want real traits, interfaces are not sufficient) the features from Scala that I need, without many of the negatives of Scala.

The biggest downsides I've encountered are around the deeper-magic Java interop, with stuff like Dropwizard's HK2 dependency injection system getting a little messy--especially around non-nullable parameters--but that stuff is, all things considered, a small part of the application.

[vvanders]

Yeah, the Kotlin inter-op is really well done. Having IntelliJ just work is always discounted more than I think people should.

I do wish Kotlin had more pattern matching semantics but that's just me being picky, it's a really nice language.

[logicchains]

>Having IntelliJ just work is always discounted more than I think people should.

To be fair, Kotlin has the advantage of being created by the company that makes IntelliJ, so it's got a big head start IDE-wise.

[aembleton]

Yes, and that is really important when it comes to language use. The more people who use the language, the more it can develop and the more libraries that can be built for it.

The other advantage of Kotlin is that it builds on top of the existing Java code base. Java has had decades to build up an ecosystem of frameworks and libraries. Kotlin builds on these with excellent interop.

[mike_hearn]

I like Kotlin too but in fairness this is a Ceylon thread ;)

From what I can see, Ceylon has a blend of strengths that should make it appeal to a certain segment of the market. On one hand, it's a fresh start and has a lot of the cleanness and terseness that I like from Kotlin. On the other, Gavin is clearly very much into type systems and type theory, and Ceylon has a more complex/powerful type system than Kotlin does.

I'll admit that I'm not totally sure whether the additional complexity is worth it, but I'm very open to being persuaded.

So it seems to me like Ceylon should appeal to people who currently like Scala or Haskell and push the type systems to the limit, but really want something without historical baggage like XML literals and other oddities. And I'm sure there are plenty of programmers like that.

The real question will be interop costs. Kotlin doesn't have any and that's the main reason people can consider it in existing projects, or introduction inside large corporations. Ceylon takes a more Scala-like approach where it has its own standard library, its own collections, etc.

[on_and_off]

not having interop costs also makes Kotlin much more realistically useable for Android apps, which could really help bootstrap its adoption.

Android could use a better language than Java 7 (or even 8) and Kotlin seems to be the only one that can easily replace it.

[gavinking]

Well the thing is that Ceylon offers fantastically smooth interop with both Java and JavaScript. That's one of it's main selling features for most people.

[mike_hearn]

Right, but by "interop" I didn't just mean the ability to call into Java code and vice-versa, although I admit that's the traditional definition. Scala can do that too.

What I meant is, Kotlin raised the bar in the interop department to new heights: it uses the same collection types, so there's no conversion between Java and some other SDK, but still manages to enhance them via compiler magic and extensions. It has an auto-converter tool so nobody has to waste time rewriting code that already works in order to use the new features. It can use annotation processors, etc.

Of course it also pays a fairly steep price to get that.

[UnFroMage]

Ceylon also has excellent Java/JVM (and JavaScript) interop. You should try it :)

[UnFroMage]

Ceylon has an IntelliJ plugin in the works that builds on the solid foundation of the Eclipse plugin, so it's picking up features really fast. It should be out in a few months.

[wtetzner]

> I think it's part of a general trend of language designers (and even language fans) not understanding what makes languages work in practice.

Maybe, or maybe not all language designers have the goal of making their language as popular as possible. A language can be popular enough, such that it has a good community and good set of libraries, without trying to get everyone to switch.

In fact, depending on what you're looking for, a language can become too popular. In this case, you start getting people in the community that detract from it, rather than contribute to it.

[lmm]

Heh, I was wondering when you'd show up :)

The study sounds interesting, though there are obvious risks to trusting self-reports, but I think you're taking a broader meaning of "familiarity" than they're using - I read the result as "programmers are likely to use a language that they know when starting a project" and I don't think you can generalize to "programmers are more likely to use a new language that's similar to a language they know" (apologies if there's something in the video that's not in the text, can't watch videos here). Certainly past a certain point the library ecosystem and pool of familiar programmers become self-sustaining, and I can well believe that such things would be the biggest factors in any given project's choice - but that can't possibly explain how a language gets to that point in the first place.

For many of us the distinguishing characteristic of Java was that it was heavily marketed, often to managers; in the early days it felt like a lot of Java users were doing so under orders. (With 1.5 and the introduction of generics the language became a lot more programmer-friendly and I think at that point it developed a lot more grassroots popularity). Being less cynical I think Java did offer something unique in the form of crossplatform multithreading with a well-defined memory model - it's easy to forget how valuable that seemed, particularly as the world moves away from conventional threading. Possibly the combination of memory-safety and claims of C-like performance was new as well? I'm speculating here as I'm not so familiar with the less popular alternatives, but I think it's not simply a case of Java being less weird or more blue-collar than Self or Modula. The popularity of Python (with weird syntax and modula-like modules) and Javascript (with a Self-like inheritance model) show that that level of weirdness is no barrier if a language has a compelling advantage.

(Likewise Go, I think - it's taking programmers largely from Python/Ruby land where it can claim a USP in the form of performance (while retaining good async support and some level of conciseness). And again, marketing seems to be somewhat involved - so often when I ask "Why did you choose Go over OCaml?" the answer is "What's OCaml?".)

I think the clearest strike against the blue-collar approach is Perl. Perl took an extremely "pragmatic" attitude to language design, adopting many "most bang-for-the-buck features" (deemphasising consistency or underlying coherence), and put a lot of emphasis on tooling and libraries with CPAN, PerlUnit and the like. And for a time it was very successful - but ultimately it became a language that was very hard to evolve, whereas other languages (in particular Python and Ruby) were able to catch up on the tooling side and their coherence gave them an advantage in the long-term. I think Kotlin is making the same... mistake would be too strong a word, Perl is a success story by many measures, but I think Kotlin is taking a very short-term strategy; it feels like every time someone figures out a clever thing you can do with implicits in Scala it gets added to Kotlin as a language feature. Everything about it feels very ad-hoc, particularly in comparison to Ceylon. So I think that while Kotlin may be a very effective language for the short term it's going to be a very brittle one that will struggle to adapt in the future.

I guess that's the closest thing to a counterexample to my original claim - a lot of people did move from Perl to Python with no USP beyond being clearer and smoother. FWIW I think the issues with Perl were deeper than those with Scala; there is some ad-hoc junk in the language (structural types should never have been added, nor should Dynamic, async/await aren't worth it...) but it feels quite peripheral, I think the core still fits together nicely. But hey, if those issues do end up forming enough of a toehold for Ceylon to take over, I'm fine with that too.

[pron]

You won't find much argument from me here, but I think I need to clarify my point. I'm not saying that blue-collar is "better" or even has better chances for adoption[1] (though I believe it does), but that when you decide to go non-blue-collar, how exactly do you do that? For example, you mention "deemphasising consistency or underlying coherence" as a bad thing. But how bad is it? We know memory safety is important. Do we know sound type systems are important to the same extent? My important meta-point was that when designing a new language for the industry, you should at least do so to solve a problem that you believe (preferably with empirical evidence) causes real pain in the real world. That's how Java, Erlang, Clojure, Rust and even Kotlin were designed (Kotlin is different because it's decided to solve a less-painful problem but for much less cost, but the same thinking is there). Are crude type systems a real problem? I don't know, but I certainly have never seen any evidence to suggest that is the case. So if you set out on this path, at least conduct a survey or research (or even your own rich experience) to convince yourself that your solving a major problem. Why? Because unlike many other tools, new languages are very expensive to adopt. The higher the cost -- the bigger the benefit you must provide.

Having said that, as you correctly note there are other paths to adoption, such as being the only language on a popular platform (JavaScript, Objective-C, Swift), or being a scripting language. Those seem to have their own rules.

[1] Although Java wasn't more marketed than other heavily-marketed alternatives at the time, and Python and JavaScript both have their own peculiar adoption stories, with JavaScript still trying to look similar to its "sister-language" Java, and then being the only choice on a popular platform, and Python taking a long time for decent adoption that even today is far from C/Java/C# levels of popularity, and either case being, or at least starting out as scripting languages for small programs -- not as languages for "serious" systems.

[lmm]

I'm not sure the evidence is any stronger. I mean, you can point to some bugs and say they were caused by a lack of memory safety, I can point to some bugs and say they were due to a poor type system.

Certainly there were problems I faced in day-to-day Java programming that were visibly due to a crude type system - NPEs, the awkwardness of working with checked exceptions (I do think Ceylon has missed a trick by not using union types to interoperate with Java checked exceptions), SQL injection vulnerabilities caused by not having a taint checker. Other problems I recognised as being due to crude typing in retrospect - the fact that missing transaction annotations could failed silently in confusing ways ( http://thecodelesscode.com/case/211 ), the ease of forgetting to close files, the way all the ways of doing async were either invisible or painfully verbose. Ultimately, everything that we resorted to (inherently non refactor-safe) annotations for. (For what it's worth, I originally picked up Scala based on my experience with the JSR308 checkers framework - the functionality was valuable but too many tools didn't interoperate with it properly, so I looked for a language that had the same thing built-in).

Any individual problem from that list you can solve at the language level - some of them have been solved in Java already, and more in the likes of Kotlin. But a powerful type system is a tool that, once it's built into the language, lets you solve them all even if you didn't think of them first. (Macros have the same kind of power, but maintainability of macros is a real problem). I don't think we'll find an example of a powerful type system being better at a specific feature any more than we'd find an example of a generic data structure being better than a specific one; the value comes in being able to do many things the same way.

Making a language that can be evolved effectively is certainly a real problem - again Perl is the clearest example, but I've also heard complaints about how long it's taken Java to implement features that have been agreed on for years (lambdas, modularization). I don't know whether it's the problem King was intending to solve with Ceylon (though I do think Rust represents a very deliberate effort at the same thing) and I don't think it's easy to prove that you have solved it short of waiting a few decades - I certainly don't have any hard evidence about what factors make a language able to evolve in the long run. Nor would it be a compelling marketing story even if you could prove it. But I think it's still worth working on.

[pron]

I really like your answer. I think it's very important to start with actual problems and then work your way towards a solution rather than vice-versa, because I've always been more troubled by all the bad things people do with rich types[1] rather than notice the importance of problems they do solve. On the other hand, any powerful general tool built into the language can be more trouble than it's worth. Macros are a good examples, but types can be worse. Macros are local to an expression, but types carry the magic with the variable. Sometimes types just carry restrictions, which is OK, but when traits/typeclasses/implicits are involved, they carry actual "bad" magic. So while other specific solutions may take time to implement, codebases live for a long time, so I'd rather wait and suffer some inconvenience, than have people turn a codebase into an unmaintainable, magical mess (which is what I've heard large companies using Scala say it's done to them). Linear types and intersection types are indeed less prone for abuse, and are mostly "restriction-only", so they seem safer, and are less likely to be abused for data types rather than cross-cutting concerns. Another approach -- my favorite -- is exactly Checker. I agree that at this point in time its UX is pretty bad, but it really doesn't have to be. It's mostly a result of the U-of-W putting more effort into the actual mechanics rather than the UX (which, IMO, is a mistake, but they're a university and naturally care more about research...). The Java annotation-processing mechanism generally works very well with IDEs.

Another question is, how bad those problems you mentioned really are. Concurrency bugs are terrible, and simple data typing bugs are usually a non-issue, but tainting/resource-use (i.e. solvable with intersection/linear types respectively) may be bad enough. I really wish someone would conduct an empirical research comprising a taxonomy of bugs, their frequency, and their cost. Such data would be infinitely more useful for language design in the industry than, say, all research going into session types (not trying to pick on session types research, but I hope you catch my meaning when I speak of actual applied utility).

BTW, I think any interesting idea is worth working on. The problem is that a language is a very expensive tool -- and a risky one -- so when it comes to industrial, rather than academic, use a lot of care should be taken when picking one, so it's better to err on the side of caution.

[1]: See Scala's collections as an extreme example. Is it really necessary to go through all that trouble of unreadable code just to save us a cast here and there that has never really bothered anyone?

[lmm]

Hmm, the reason I find types clearer than macros is that they do apply everywhere, so there's consistency across a codebase. <- will always mean a call to flatMap - which might do something different in one place or another, but only in the same way as a normal polymorphic method call - whereas different macros might use it to mean unrelated things. I think IDEs handle them better too - you can refactor in the normal way, you get types on mouseover. Last time I tried to use Checker it was similar - the IDE has good support for running the checkers, but it can't and won't apply the right annotations when extracting a method, or show you the checker-inferred types of local variables.

I do think implicit conversions allow too much; IMO we need typeclasses and extension methods. So my ideal design for a language would include implicit parameters but not implicit conversions, and instead have dedicated support for extension methods. That said we do need to be able to use typeclasses without disrupting method application syntax (i.e. support the use case that means Spray prefers the implicit-conversion-based "magnet pattern" rather than a typeclass).

More generally I think there will be languages that allow too much and languages that ban too much, and we need languages on both sides of the line to allow us to converge on something better. Most places I've worked have had a strong code review/pull request culture which may have coloured my experience - I just haven't seen the Scala maintainability problems you talk about.

(If I thought the complexity of collections was necessary to avoid casts then I would defend it - casts can fail and so it's very nice to have a codebase where they're banned entirely (enforced with wartremover). But I don't think it is - FWIW Odersky's asked for proposals for scala collections in version 3, and I think there's a substantial push towards at least some simplification - perhaps replacing CanBuildFrom with a 1-parameter typeclass that would retain most of the power and simplify the code. The showing-off cases like BitSet#map would require one more method call, but no casts)