[packetlost]

> I lately start to think homoiconicity is really a wart.

I sorta agree. The simplicity of the syntax and representation makes it particularly amenable to dynamic modification above basically every other language though. There's a bunch of other properties that make this to case though, such as a very dynamic type system, first-class functions, extremely simple syntax, etc. so it's really a combination of a bunch of factors.

That being said, I think homoiconicity is actually a useful feature, but runtime macro expansion is the dangerous part. What I'd really like to see is a Lisp with very well defined execution orders. That is, macros must be expanded at compile time, and with clear symbols for defining what runs and when. I'm not talking about something like `(macroexpand ...)`, more like `(comptime (my-macro ...))`... `(define (my-func) (my-macro+ 'a 'b 'c))` where it's explicit that a macro executes at compile time, and usage of that macro must be denoted with a special character (`+`, in my example).

I think a general purpose macro language is only as useful as the average code-gen/templating language. It's just string manipulation, which can be harder to reason about than true metaprogramming and might result in some ugly/verbose output and having to context-switch between 2 entirely different languages often. What's particularly powerful about Lisp macros is that usage of them doesn't look any different than a normal application usually, and writing a macro is only marginally different than writing a normal function.

[hzhou321]

> I think a general purpose macro language is only as useful as the average code-gen/templating language. It's just string manipulation, which can be harder to reason about than true metaprogramming ...

I would like you to reconsider. Predicting a program output is hard. So in order to comprehend a macro programed as code, one need run the macro in their head to predict its output, then they need comprehend that output in order to understand the code. I think that is unreasonable expectation. That's why reasonable usage of meta-programming is close to templating where programmer can reason with the generated code directly from the template. For more higher-powered macros, I argue no one will be able to reason with two-layers at the same time. So what happens is for the programmer to put on his macro hat to comprehend the macro, then simply use a good "vacabulary" (macro name) to encode his comprehension. And when he put his application programming hat, he takes the macro by an ambiguous understanding, as a vocabulary, or some one may call it as a syntax extension. Because we need put on two hats at different time, we don't need homoiconicity to make the two hats to look the same.

[packetlost]

Or you do:

``` (require (ast macroexpand))

(display (macroexpand my-macro arg1 arg2)) ``` and call it a day.

My argument isn't that the context-switch isn't there, it's that the context switch will happen regardless and having to think in 2 different languages is more mental overhead than is necessary. I do agree that homoiconicity is not a requirement, but it is nice to be able to do it all in one go and with no additional tooling, editor, etc.. In reality, a sizeable chunk of Lisp programmers are executing their code in a REPL continuously as part of their core development loop, there's no tooling (context) switch, no language context switch, and barely a macro vs application code context switch.

To illustrate, Rust macros are basically that. They have a substantially different syntax to normal Rust code that make it very difficult to quickly grok what is going on. It's a net negative IMO, not a positive.

[hzhou321]

> To illustrate, Rust macros are basically that. They have a substantially different syntax to normal Rust code that make it very difficult to quickly grok what is going on. It's a net negative IMO, not a positive.

Yeah, more like a syntax extension than macro. But I am saying that you need both. Some time you need powerful macro ability to extend the language. Sometime you just need templating to achieve the expressiveness. With LISP, I get it that you are programming all the time, never templating, right? But I guess you only appreciate templating when you use your macro system as a general purpose system . The benefit of general purpose macro systems is you only learn one tool for all languages, rather than re-learn the individual wheels. And when you judge a language, you no longer bothered by its syntactic warts because you can always fix the expressive part with your macro-layer.

[hzhou321]

> That being said, I think homoiconicity is actually a useful feature, but runtime macro expansion is the dangerous part.

Practically, why would you ever want a runtime macro expansion?

[packetlost]

Well, part of the problem is is passing around quoted forms (ie. `(quote ...)` or in may Lisps, `(...)). If that form contains a macro, and a sizeable chunk of Lisps stdlibs are macros, you probably need to implement runtime macro expansion at least partially. So in order to avoid implementing runtime macro expansion, you need to break the semantics of what a quoted form is, or disallow them entirely. The implementation for the former could get really complicated depending on the cases you want to support, resolving symbols in the corresponding scope comes to mind as being particularly challenging. Removing quoted forms entirely just really isn't an option, it's required for one of the most powerful parts of Lisps in general: built-in syntax templates. So we're back to breaking the semantics of quoted forms, which I think can be done reasonably, if not difficult to implement.

[hzhou321]

In another word, it (to have runtime macro expansion) is a side effect, a compromise, a wart, rather than a design goal, right?

[packetlost]

It can certainly be a design goal if you want it to be. Sometimes truly dynamic programming is what you want and need. I, personally, wouldn't want to write any code like that because I mostly write software for other people that runs while they aren't looking, not for myself that runs when I execute it. Lisps are popular in academic settings where the program behavior is the topic of interest and maximum flexibility is a tool to accelerate progress. These types of scenarios usually have the person who wrote the code directly involved in it's execution, as opposed to service development where you want to be more sure it'll actually work before you deploy it.

[JonChesterfield]

Most common case is probably trying to pass `and` to something that applies it to a list, `fold` or `reduce` or similar, and being told some variant of "no deal, and is a macro, not a function" with workarounds like `(reduce (lambda (x y) (and x y)) list))`

[kazinator]

Mainly, because it's always run-time. Your program's compile-time is the compiler's run-time. Compilers can be available in a shipped application.