[sarmad]

I am. I am working on a new programming language designed for the next generations. The language re-imagines the role of the compiler from being a monolithic static blackbox that converts source to executable, into being an open dynamic system the manages the language syntax and compilation process, but leaves the final syntax and actual compilation to hot pluggable libraries. Why? To make the language dynamically upgradable, and on a per-project basis. This is the only way to make a language future-proof. More details: http://alusus.net http://alusus.net/overview P.S. the project is still at a very early stage, but a proof of concept is there.

[dTal]

Oh man, this is exactly what I've been thinking about lately (though I think it should be built on something like Scheme). The more research I do, the more it seems like a Hard Problem. In a nutshell, how do you handle inherently ambiguous composed grammars? Do you see your system being flexible enough to be able to essentially "behave like" arbitrary existing languages? It would be very easy to port things to it if this were the case.

Anyway, good luck!

[sarmad]

It's not my goal to make the system able to parse arbitrary existing languages. The parser can deal with ambiguity by branching parsing states, but despite that it still won't be easy to parse inherently ambiguous grammars, and even if that was easy to do I still would push against it as that would be in conflict with the language's goal of unifying different programming areas under one language with consistent syntax. It wouldn't be much useful language if your code is written like c++ in one place and like, say, Ruby or SQL in another. You need your code to be consistent and the language to be consistent as well, otherwise learning the language would be difficult and understanding the fragmented user code would also be difficult. The main benefit behind being able to parse existing languages is for porting existing code, but that can be done using external conversion tools.

[teraflop]

I read the links; your language looks interesting, but could you expand a bit on what makes it a new approach? User-programmable syntax is an old idea; for instance, it's been a core feature of Scheme for 30+ years.

[sarmad]

My language is not only about user defined syntax; it's also about user defined code generation. For example, you can define your own syntax that compiles directly into a DB stored procedure or into GPU shader, or into whatever technology that might be invented in the future, like for example quantum computing. In fact, it's less about user defined syntax than it is about user defined code generation as I am trying to limit new syntax into certain patterns rather than leave it in the wild.

[gnaritas]

That's what macros are in Lisp and Scheme.

[sarmad]

Does scheme give you access to the compiler's internal data structures? Can you for example write an extension that scans through the compiled code looking for, say, for loops and replacing them with something else?

[gnaritas]

Macro expansion happens just before compilation; you have the chance to alter the source code as it's being read by the reader just before compilation.

But I see what you're getting at; you mean global modifications to the compiler to interpret the whole language differently. A macro allows you to make syntax, but it doesn't change the meaning of other syntax elsewhere in the program; only allows the programmer to add syntax to the language.

[kazagistar]

Macros generate assembly?

[gnaritas]

Macros generate code, any code you want, including inline assembly if the scheme supports inline assembly, if not no.

[dTal]

And yet Scheme hasn't turned into something like that. There's a limit to how radical a syntax you can achieve using (define-syntax), and there's no provision for modularity. Worse still, syntax definitions are "imperative" - it's really hard to see what they do by looking at them.

I'd like to see a system where you have tiny, composable pieces of syntax written declaratively in a domain specific language that's as terse and general as possible - stuff like "a[n] -> (list-ref a n)" and "a..b -> (iota a b)". You want to make it as easy as possible to port language concepts to the system. Most of the wacky semantics you can get in other languages can be had in Scheme through some sort of library already, they're just usually kind of a pain in the ass to use without the syntax. Then making the language of your dreams is just a matter of enabling all the syntax you like. You'd need some kind of mechanism for handling grammar conflicts.

Of course, it's really hard. "Syntax" masks a lot of complexity. A feature I want, for instance, is to have the whole language behave like a CAS, where instead of bombing out on an undefined symbol it just computes as much as it can symbolically and hands it back to you e.g. "x+2+3" returns "x+5" if x isn't defined. That sort of thing has been done in Scheme of course, but it's hard to imagine a syntax specification language that would let you just turn that on as a single "feature" without massively affecting basic things like expression parsing grammar.