[kazinator]

I have developed a very nice Lisp dialect from the ground up, in which macros do all these things:

* completely implement the system of "syntactic places": the ability to mutate variables, mutable slots of various kinds on various objects, and user-defined such things. The only API underneath consist of ad hoc functions to update specific kinds of places.

* implement the syntax of the object system: structure defining via defstruct, instantiating objects with new and so on: over top of an API consisting of functions to manipulate the object system.

* various control structures.

* the op macro for currying / partial application.

* the FFI language for defining foreign types, and its operators like typedef, sizeof and offsetof.

* the awk macro: a fully-loaded, "Lispified" version of the Awk language.

* generator-like programming interface (featuring yield with bi-directional communication of values) implemented over top of delimited continuation primitives.

Many more. Macros provide a catalyst for language research. New syntax can be added to existing syntax without any conflict, and without having to muck around with a parser. The new syntax doesn't even have to be loaded into memory if it's not being used.

"Macros don't do much; I don't need need to write macros" ignores all the macros that have been already provided and the benefit they bring. You can't be using existing macros left and right while claiming that they are not all that necessary.

You may feel that way when using a language that has no macros. But the syntax of such a language is effectively a macro expander. A Yacc rule (or what have you) which matches some symbols on the left and produces a piece of AST in its action body is effectively a macro. Someone else wrote that syntax and you benefit from it, just like you benefit from existing macros in a Lisp. But that syntax hinders development of that language; it is ultimately detrimental to that language. Because new versions of the language struggle with the integration of new syntax, they create backward-compatibility wrinkles.

With macros, if I have some piece of code that uses syntax available only in the latest and greatest version of the language (not what I'm using in production), I may be able to just write the macro to simulate the feature well enough to get the code working. Code relying on hard-coded new syntax has to be rewritten not to exhibit a syntax error with the older language.

The ability to define macros can really help work around bugs in existing macros. If hard-coded syntax is broken, all you can do is code around it with different syntax. If a macro is broken, you can write a macro just like it (only with a different name) or perhaps even redefine it. The code doesn't have to change at all. Eventually when upstream fixes the bug, the workaround is removed and that is that.

Since macros are ordinary identifiers, they can be namespaced. Just because you introduce a new control construct called wonk doesn't mean that wonk is now a reserved token in a syntax, which breaks everyone who used wonk as a variable name. Syntax-in-a-package (rather than in a global parser) is awesome!

If Python had macros, Python 2 to 3 migration would be a non-issue. It wouldn't be a topic of discussion that non-Python-programmers know about.

[irahul]

> Someone else wrote that syntax and you benefit from it, just like you benefit from existing macros in a Lisp.

Are you following the discussion? The whole discussion arose from somehow macros make up for non-existence of libraries in Racket. Someone else wrote all those libraries for popular languages and I benefit from those existing libraries. Macros do fuck all if I have to wrap them myself. And even if I have to, most of the times it will be way simpler to wrap it in Python/Ruby.

You wrote a novella on your hypothetical macro use cases. I wrote I know what macros are. I don't know why you are throwing that at me.

> If Python had macros, Python 2 to 3 migration would be a non-issue. It wouldn't be a topic of discussion that non-Python-programmers know about.

This is what I mean when I say macros are a meme.

No, it won't have helped, at all. The changes in python 3 break the existing python 2 code.

> For example, this no longer works: k = d.keys(); k.sort(). Use k = sorted(d) instead (this works in Python 2.5 too and is just as efficient).

> The ordering comparison operators (<, <=, >=, >) raise a TypeError exception when the operands don’t have a meaningful natural ordering

> builtin.sorted() and list.sort() no longer accept the cmp argument providing a comparison function.

> The biggest difference with the 2.x situation is that any attempt to mix text and data in Python 3.0 raises TypeError, whereas if you were to mix Unicode and 8-bit strings in Python 2.x, it would work if the 8-bit string happened to contain only 7-bit (ASCII) bytes, but you would get UnicodeDecodeError if it contained non-ASCII values.

I could go on, but you get the idea.

[kazinator]

> No, it won't have helped, at all. The changes in python 3 break the existing python 2 code.

In a Lisp setting, the 2 code could be supported in an environment where it is not broken, in the same image in which 2 code is running. It could be redirected to use library symbols from a 2 package. E.g. if the hypothetical floop syntax changed there could be a ver2:floop and ver3:floop.

If necessary, the entire 2 implementation could be separately provided via this backward compatibility mechanism. Yet, 2 and 3 code could be freely mixed even in the same function. E.g. an instance of ver2:floop could nest inside ver3:floop.

[irahul]

I don't follow.

> if the hypothetical floop syntax changed there could be a ver2:floop and ver3:floop.

dict.keys() returned a list earlier, now it returns a "view". dict already had keys() and iterkeys(). Is that what you were referring to by your ver2:floop and ver3:floop? They changed keys() to act essentially like iterkeys() and removed iterkeys() because per them, this is the correct behavior.

Most changes in python 3 won't be helped by macros. Macros would have helped if they were adding say pattern match. But even then, it doesn't really matter much. If python itself is adding pattern match, they can write it in C, or python, or write a macro(if python had it) - it would have been all the same to me.

I am not saying macros aren't useful. I am saying macros are useful in limited circumstances and existence of macros won't have done anything for python 2 to 3 migration.

[kazinator]

Can you name anything that is useful in unlimited circumstances?

[irahul]

You are just deflecting now. You made the claim that somehow macros would have made the python 2 to 3 migration easy. You are yet to demonstrate which of the migration issues would have been solved by macros.

[kazinator]

dict.keys() returning something different from what it used to isn't a syntactic change; it's a change in function behavior. Since macros handle syntax, they aren't applicable to a function semantics change.

What we might do here is to use two different keys symbols in different packages. We can have a ver2:keys and ver3:keys and control which of these dict.keys() uses in some scope. But that's not macros.

The point is that with macros, we can do the same thing with the syntax of the language; it's a library feature that we can subject to versioning. Old syntax can be supported side by side with new syntax. So then since we have a way to have old syntax and old API semantics, which is pretty much everything, we can have a nice migration path.