[pron]

I think there are.

Java was not the first popular programming language, and software has been around long enough to know how language adoption works. The secret is incrementalism. If you want to consider the Java world, then Java was an incremental change in syntax over C++ and a revolution in the runtime, while other JVM languages might be an incremental change in use of the runtime but a revolution in syntax/concepts.

So incrementalism is one way. The other is picking a niche. Some languages are widely used in certain industries because they are particularly well suited to some use cases. That kind of success is often enough, but sometimes a language might expand from its original niche after having achieved success there.

But in many cases, the sentiment you're reeling against is actually correct. The fact that language B is better than language A does not always (or even often) mean that the best course of action for the organization is to adopt language B. Switching a language has a very significant cost, and it's a change that a company should never consider more often than once every five years and optimally not more than once a decade, unless language B is so advantageous that it would be stupid not to switch. Well, that's assuming the choice of language A was wise in the first place.

Because the switching cost is high, there's a very high bar to competing successfully, as there well should be.

[loup-vaillant]

This whole "competing languages" is backwards.

We write our own application code. We often write our own (specialized) library code. Sometimes, we even write our own little frameworks. But we very rarely write our own languages. There's a wall somehow between languages and everything else: the language is something you chose. Everything else is something you might build —or at least adapt to your ends.

Why? This is beyond silly.

Take one example: Closures in Java. Closures are very useful. If you put garbage collection in your language, but forget about closures, you fail at language design forever. Java made this mistake, and forced us to use anonymous inner classes, and other equally horrible workarounds instead. Java need closures and we knew that for years.

What did we do? We waited for the landlord.

We didn't have to. I'm sure some of you have heard, there's this nifty concept called "source to source transformation"[1]. Here's how it translate in Java: you take source code that is like Java, but is not quite Java. Feed that into your compiler, output Java code, and voilà, you have modified the language. In the specific case of closures, it would mean invent a syntax for closures, and compile that into those ugly anonymous inner classes.

Some people probably have done that, and I just don't know about them. Anyway, it seems the technique didn't get any traction. (The only example I know of is here[2]. Skip to page 6 and 7. Then read the whole thing.)

I don't know about you, but the next time someone forces me to use a language that sucks, I'm going to at least write a preprocessor, and modify the damn language. In other words, I can write LISP in any language, and I will. And when I'm done, even COBOL won't suck.

[1]: http://www.youtube.com/watch?v=EGeN2IC7N0Q

[2]: http://www.cs.indiana.edu/~dfried/dfried/dfried/mex.pdf

[judk]

The gatekeepers at an organization refuse to allow devs to submit unreadable machine generated source code, and won't allow the fragmentation of multiple original source languages.

But 1000 ad-hoc human written micro a frameworks and spaghetti approximations or workarounds to closures etc? No problem. Massive LoC throughput, solving "complex problems" as demonstrated by code base size...

[pron]

I often find that the people deriding the "gatekeepers" have never been gatekeepers themselves. Obviously, every developer should do whatever the hell they find most productive, because developers are known to have such incredible foresight, they never just pick tools they feel like using for no good reason, and they always consider the grand-scale effect of their decisions on the organization as a whole.

[loup-vaillant]

I have a feeling that the "grand-scale effect" you speak of is overrated. Any sufficiently big organization will be fragmented into groups that hardly talk to each other. At that point, if you want code that's usable by several groups, you must publish it as if it were Open Source (and have it compete with actual Open Source code). That is so much effort that most simply won't do it. It may not be even worth the trouble since the different groups are probably working on different software anyway.

While it makes sense to follow some unified standard as a team, it makes much less sense across teams. With few exceptions, there is no "grand scale".

[loup-vaillant]

I am soo going to use that argument.

[tel]

Languages have a LOT of ecosystem demands that frameworks and libraries do not. DSLs (especially embedded ones) save on those demands and do get written all the time. That said, I'm of the belief that they should get written more, still.

[loup-vaillant]

> Languages have a LOT of ecosystem demands that frameworks and libraries do not.

I have the feeling that you know something I don't. What do you mean by "ecosystem demand"? Could you please name three?

[tel]

I'm sure it's more my poor choice of word, sorry about that. I just meant that if you're starting from scratch on a language there's a fairly large amount of non-useful stuff that must be done before it becomes valuable.

I'm basically assuming that the only reason why you'd have enough business value in creating a new language comes from it either modeling something interesting or having a different semantic focus. Everything else is just wasted time then.

Some examples:

1. Documentation. How do people know what your language means? How do they know what happens when it breaks?

2. Shaking out edge cases (much harder to do than debugging a program!)

3. Syntax. Picking it. Parsing it. Ensuring it's complete and doesn't have weird edge cases.

4. Compiler. Runtime. Parallel (!) runtime? Platform independence? Efficiency?

5. Static analysis (type checking, memory analysis, nulls)

6. Managing modules/files/packages

7. Foreign calls

8. Dependency management

All of these are fun in their own right, but most businesses would have a hard time arguing that they're worth working on unless they've already got a language tied into their core value proposition. It's also important to mention that there exist tools to make some of them easier (bnfc, llvm, &c).

But if you just start with an embedded DSL you can get right to the modeling or semantic issues by piggy-backing on the host language for (most of) 2, 3, 4, 5, 6, 7, and 8. If you prove that there's something valuable there and the host language is slowing you down then you can start to unembed it by building the "whole ecosystem".

[loup-vaillant]

We don't have to chose between embedded DSL (meaning, a kind of fancy library), and a fully fledged external DSL. I was advocating source-to-source transformation, which I think is a nice compromise between the two: it feels almost like an external DSL, but doesn't require much more work than an internal one.

The way I see it, source-to-source transformation helps a lot with 2, 4, 5, and 7: we can rely on the host language for these. Actually, 5 is even better than an embedded DSL: you can implement additional checks. As for 6 and 8, you will generally avoid the issue altogether by letting your host language deal with them.

An example of source-to-source transformation at work is OMeta. The thing compiles itself to JavaScript, and its source code takes about 400 lines of JavaScript and OMeta. I'm currently working on a clone for Lua, and my prototypes so far are under 300 lines.

I agree that picking up a syntax is hard (parsing it is trivial). But with enough feedback, you can make a decent one. Just make sure to tell everyone that the first 5-10 versions of your language are all pre-alpha, and totally incompatible with each other. And source to source transformation isn't about building a whole new language. Most of the time, it is about extending the host language. This means much less syntax to worry about in the first place.

Actually, if you're smart about it, you can often write most of the semantics of your language in a library, then add a light layer of syntax sugar on top of it. That's what I did when implementing a PEG syntax for Haskell[1]. It's just syntax sugar over Parsec.

[1]: http://loup-vaillant.fr/projects/metacompilers/ (bottom of the page)

---

Now back to my Java example: It was just a matter of adding a nice syntax sugar over anonymous inner classes, and maybe allow a syntax for calling objects as if they were functions, C++ `operator()` style (Even so, writing obj.app(args) isn't that cumbersome). Not much work, really.

[tel]

Source-to-source is an interesting example. I think Coffeescript has proven it's pretty possible to get a long way there, though it's undoubtable that it still has some syntactic and semantic design issues.

I think another important example of source-to-source is Lisp macros. Super powerful and when used well and marketed well they can really transform what Lisp means. I also think that when marketed poorly they lead to an explosion of competing "languages".