[WalterBright]

I'm referring to both syntax based (AST) macros, and text based (preprocessor) macros. The latter, of course, are much worse.

An example of the former is so-called "expression templates" in C++. I've seen them used to create a regular expression language using C++ expression templates. The author was quite proud of them, and indeed they were very clever.

However nice the execution, the concept was terrible. There was no way to visually tell that some ordinary code was actually doing regular expressions.

C++ expression templates had their day in the sun, but fortunately they seem to have been thrown onto the trash pile of sounds-like-a-good-idea-but-oops.

(I wrote an article showing how to do expression templates in D, mainly to answer criticisms that D couldn't do it, not because it was a good idea.)

[d110af5ccf]

> Anytime macros are used for metaprogramming, it's time to reach for a more powerful language.

> I'm referring to both syntax based (AST) macros ...

This surprises me greatly. Various lisps are among the most powerful languages I know of and a large part of the reason is macros coupled with their ability to execute arbitrary code at compile time (which itself uses additional macros, which in turn invoke more code, and so on). What's your take on this?

(Continuations are also pretty nice ...)

[WalterBright]

I'm a noob when it comes to Lithp. But I'm told what happens with their macros is the language is fairly unusable until you write lots of macros. The macros then become your personal undocumented wacky language, which nobody else is able to use.

I've seen this happen with assembler macro languages, too.

> most powerful

It's like putting a 1000 hp motor in a car. It's main use is to wreck the car and kill the driver.

BTW, D is the first language of its type (curly brace static compilation) to be able to execute arbitrary code at compile time. It started as kind of "let's see what happens if I implement this", and it spawned an explosion of creativity. It has since been adopted by other languages.

[d110af5ccf]

> the language is fairly unusable until you write lots of macros

This is not (typically) the case. It would be like saying that you need to write lots of templates to get things done in D. Metaprogramming is certainly very nice to have but it's not a requirement for the vast majority of tasks.

It's important to note that Lisps are an entire family of languages; some implementations are batteries included while others are extremely minimal. Where things can get a bit confusing is that many macro implementations are so seamless that significant pieces of core language functionality are built in them. Schemes tend to take this to an extreme, with many constructs that I would consider essential to productive use of the language provided as SRFIs.

> macros then become your personal undocumented wacky language

That's Doing It Wrong™. You could as well argue to remove goto from a language because sometimes people abuse it and write spaghetti. C++ has operator overloading. D has alias this. If (for example) a DSL is the appropriate tool then being able to use macros to integrate it seamlessly into the host language is a good thing.

[WalterBright]

> it's not a requirement for the vast majority of tasks.

Right, but the temptation to do it is irresistible.

> That's Doing It Wrong™

Of course it's doing it wrong. The point is, that seems to always happen because the temptation is irresistible.

> goto

I rarely see a goto anymore. It just doesn't have the temptation that macros do.

> alias this

Has turned out to be a mistake.

> integrate it seamlessly into the host language is a good thing

Supporting the creation of embedded DSLs is a good thing. Hijacking the syntax of the language to create your own language is a bad thing. I've seen it over and over, it never works out very well. It's one of those things you just have to experience to realize it.

D's support for DSLs comes from its ability to manipulate string literals at compile time, generate new strings, and mixin those strings into the code. This is clearly distinguishable in the source code from ASTs.

[d110af5ccf]

Fair enough, it seems we have very different philosophies on this particular issue. Without oft maligned features such as alias this and operator overloading I never would have given D a try as an alternative to C++.

I'd like to suggest that I think you might be missing some perspective here. You say that misuse of macros always happens and that you've seen it over and over. Yet if you explore the Scheme ecosystem you might notice that significant parts of any given implementation often take the form of macros. Racket in particular fully embraces the idea of the programmer mixing customized languages together and while examples of bad code certainly exist it seems to work out quite well on the whole.

To be clear, I do appreciate having easy access to tools that are simple and safe. I just also like having seamless access to and interop with a set of powerful ones that don't try to protect me from my own poor decisions. I shouldn't need to do extra work to make use of an alternative more powerful tool for a small part of a project. At that point it becomes very tempting to drop the safer tool altogether in favor of the more powerful one just to avoid the obviously needless and therefore particularly irritating overhead.

I much prefer the approach of providing limited language subsets that can be opted into and out of in a targeted manner. Having the compiler enforce a simple one by default provides a set of guard rails without getting in the way when it matters.

If I could write the majority of my code in something resembling Go and just a small bit of it in an alternative dialect with expressive power comparable to Common Lisp that would be ideal. To that end, I'm a huge fan of features like @system, @safe, and @nogc in D while very much disliking the need to use string mixins to write a DSL, the various restrictions placed on CTFE behavior, and other similar things.

[nextaccountic]

> BTW, D is the first language of its type (curly brace static compilation) to be able to execute arbitrary code at compile time. It started as kind of "let's see what happens if I implement this", and it spawned an explosion of creativity. It has since been adopted by other languages.

I don't know much about D compile time evaluation. How is it better than macros/templates?

Also, what do you think about Haskell's and Rust's approach of generics with typeclass/trait bounds?

[d110af5ccf]

> D compile time evaluation. How is it better than macros/templates?

CTFE isn't better than templates, it's a completely different tool. CTFE computes a result at compile time as though you had written a literal in the source code. Templates generate blocks of specialized code on the fly based on various parameters (typically types). They solve different problems.

[nextaccountic]

I think that Walter was trying to say that D gets by doing the same things that people use macros for, but without macros. I interpreted this as meaning that D uses CTFE in those same situations. Am I wrong?

What does D do, then?

[d110af5ccf]

I think he was saying that sufficiently powerful languages provide enough features that you won't need macros in the first place. If you feel that you do for some reason, go find a more powerful language instead.

> What does D do, then?

Most mainstream languages, D included, very intentionally don't provide any features that could potentially be used to extend the language itself on the fly. (At least not in a straightforward manner. Obviously the C preprocessor kind of sort of facilitates a bit of this.)

As a counterexample, Rust does provide some of this in the form of procedural macros but doesn't provide (to the best of my knowledge) an equivalent to Lisp reader macros.

[WalterBright]

I wish you luck with it. If you send me an email, when I find what I wrote about it I'll pass it along to you.

[nextaccountic]

I think you replied the wrong comment

[shaklee3]

Can you link to it? We're using expression templates on a new library and I find it useful.

[acehreli]

I don't know what code Walter is referring to but I had similar experiences during a presentation of the Boost Spirit library (at our Silicon Valley C++ Meeting; must be about a decade ago now).

The speaker was proud and beaming for showing how powerful C++ is and the audience was in awe.

I was incredulous! Jaw open! The "solution" was horrible with a bunch of workarounds for a bunch of shortcomings. It was a "the emperor does not have cloths" moment for me.

Boost Spirit may be better today with newer C++ features; I don't know.

[shaklee3]

That's kind of the key. We require c++17. Without that it gets extremely ugly and more verbose.

[WalterBright]

Oh man, you sure know how to drive a stake in my heart! What have I done! Some curses should just not be uttered. I'm not sure where it is on my backups.

But it was done just like you'd do it in C++. The same thing, just with D syntax.

[shaklee3]

We're making a numerical computing library that expression templates are used for compile-time evaluation of expressions. It's a very niche hpc library, so it might be one of the few places it's appropriate.

[WalterBright]

I wish you luck with it. If you send me an email, when I find what I wrote about it I'll pass it along to you.

[1980phipsi]

They probably aren't the only people interested in that...