[tel]

Languages have a LOT of ecosystem demands that frameworks and libraries do not. DSLs (especially embedded ones) save on those demands and do get written all the time. That said, I'm of the belief that they should get written more, still.

[loup-vaillant]

> Languages have a LOT of ecosystem demands that frameworks and libraries do not.

I have the feeling that you know something I don't. What do you mean by "ecosystem demand"? Could you please name three?

[tel]

I'm sure it's more my poor choice of word, sorry about that. I just meant that if you're starting from scratch on a language there's a fairly large amount of non-useful stuff that must be done before it becomes valuable.

I'm basically assuming that the only reason why you'd have enough business value in creating a new language comes from it either modeling something interesting or having a different semantic focus. Everything else is just wasted time then.

Some examples:

1. Documentation. How do people know what your language means? How do they know what happens when it breaks?

2. Shaking out edge cases (much harder to do than debugging a program!)

3. Syntax. Picking it. Parsing it. Ensuring it's complete and doesn't have weird edge cases.

4. Compiler. Runtime. Parallel (!) runtime? Platform independence? Efficiency?

5. Static analysis (type checking, memory analysis, nulls)

6. Managing modules/files/packages

7. Foreign calls

8. Dependency management

All of these are fun in their own right, but most businesses would have a hard time arguing that they're worth working on unless they've already got a language tied into their core value proposition. It's also important to mention that there exist tools to make some of them easier (bnfc, llvm, &c).

But if you just start with an embedded DSL you can get right to the modeling or semantic issues by piggy-backing on the host language for (most of) 2, 3, 4, 5, 6, 7, and 8. If you prove that there's something valuable there and the host language is slowing you down then you can start to unembed it by building the "whole ecosystem".

[loup-vaillant]

We don't have to chose between embedded DSL (meaning, a kind of fancy library), and a fully fledged external DSL. I was advocating source-to-source transformation, which I think is a nice compromise between the two: it feels almost like an external DSL, but doesn't require much more work than an internal one.

The way I see it, source-to-source transformation helps a lot with 2, 4, 5, and 7: we can rely on the host language for these. Actually, 5 is even better than an embedded DSL: you can implement additional checks. As for 6 and 8, you will generally avoid the issue altogether by letting your host language deal with them.

An example of source-to-source transformation at work is OMeta. The thing compiles itself to JavaScript, and its source code takes about 400 lines of JavaScript and OMeta. I'm currently working on a clone for Lua, and my prototypes so far are under 300 lines.

I agree that picking up a syntax is hard (parsing it is trivial). But with enough feedback, you can make a decent one. Just make sure to tell everyone that the first 5-10 versions of your language are all pre-alpha, and totally incompatible with each other. And source to source transformation isn't about building a whole new language. Most of the time, it is about extending the host language. This means much less syntax to worry about in the first place.

Actually, if you're smart about it, you can often write most of the semantics of your language in a library, then add a light layer of syntax sugar on top of it. That's what I did when implementing a PEG syntax for Haskell[1]. It's just syntax sugar over Parsec.

[1]: http://loup-vaillant.fr/projects/metacompilers/ (bottom of the page)

---

Now back to my Java example: It was just a matter of adding a nice syntax sugar over anonymous inner classes, and maybe allow a syntax for calling objects as if they were functions, C++ `operator()` style (Even so, writing obj.app(args) isn't that cumbersome). Not much work, really.

[tel]

Source-to-source is an interesting example. I think Coffeescript has proven it's pretty possible to get a long way there, though it's undoubtable that it still has some syntactic and semantic design issues.

I think another important example of source-to-source is Lisp macros. Super powerful and when used well and marketed well they can really transform what Lisp means. I also think that when marketed poorly they lead to an explosion of competing "languages".