[patrec]

Well, certainly no one seems to understand how e.g. syntax-case works. But my impression is that macro hygiene in itself is a solution looking for a problem. The key advantage e.g. racket's macro system has over clojure or common lisp is not hygiene but being sufficiently well structured and rich to allow proper tooling. Good error messages with accurate locations >> macro hygiene.

[deckard1]

> macro hygiene in itself is a solution looking for a problem

No. Unless you're not familiar with Lisp-1 vs Lisp-2. In Scheme, you would have to GENSYM every variable in addition to every function you call within a macro. Whereas in Common Lisp you just need to GENSYM the variables. That's the real reason Scheme doesn't use DEFMACRO.

I'm not personally a fan of any hygienic macro system because learning a new language defeats the purpose and elegance of Lisp macros in the first place. But then again, outside of personal projects and academic exercises, no one should be using macros. Messing with fundamental semantics of a language while other developers are working on the same project will certainly make you a ton more enemies than friends. I still have a grudge against the guy that used Ruby's method_missing and I spent an entire day hunting down a method that didn't exist. When I figured it out, I don't think I've ever been so pissed at someone before.

[patrec]

> No. Unless you're not familiar with Lisp-1 vs Lisp-2.

Just because "classical" defmacro is mildly awkward to use in Common Lisp and much more so in scheme does not mean that the only viable alternatives are R*RS style hygienic macros. Look at how e.g. clojure does it, it has a simple and perfectly adequate solution (a concise gensym notation, basically). As far as hygiene is concerned. Not so much in terms of generating good error messages.

[fiddlerwoaroof]

The problem with Ruby’s method missing is mostly that Ruby development generally doesn’t happen in an environment where discovering the existence of that method is easy. Macros are even better and tools like the slime macrostep expander make them relatively easy to deal with.

[akiselev]

I think in theory they’re orthogonal but in practice the two go together. It’s all fine and dandy when you’ve got a restricted set of battle tested macros from a single library interacting in real world code, but it rapidly breaks down when you’ve got application authors of various skill sets all contributing their own macros because the dare not touch the ones that came before. Macro hygiene provides the equivalent of type level guarantees to metaprogramming scope.

[patrec]

Is this based on conjecture or personal experience from working on multi-dev projects with both hygienic and unhygienic macro systems?

I lack such comparative experience but my guess would be that if your org's development approach is such that inexperienced people churn out dodgy macros without more expert review you are screwed and having a hygienic macro system will not save you. I'm very sceptical hygienic macros are at all comparable in utility to what a type system provides for multi-dev projects of any size. These benefits of types at scale are fairly massive and apply to basically any code. In contrast macros should make up a tiny amount (much less than 5% certainly) of code in a non-trivial application code base which means careful review is not a big overhead (and in my experience competent programmers do not introduce a lot of hygiene problems). So to be equivalent, in those instances where you do get a benefit from hygiene, that benefit would need to be absolutely massive compared to the average benefit you get of types.

[mpweiher]

And maybe Macros aren't the right kind of metaprogramming mechanism...

[patrec]

Why not?

[kazinator]

Macro hygiene is a solution to the problem of functions being in the same namespace as variables, together with standard, oft-needed library functions having short names that are easily chosen as variable names.

[patrec]

Well, that's the standard Common Lisper opinion. I don't buy it though -- I don't get the impression that it's a problem in clojure (which does not have scheme-style hygenic macros, but shares the single namespace for variables and functions) either.

[kazinator]

The hygiene problem has multiple aspects. One issue is that the user's code can bind identifiers which the macro expansion expects to be predefined.

In a Lisp-1, we have to worry about this:

  (let ((list ...))
    (mac ...)) ;; expansion of mac wants to call (list ...)

Since the macro is not defining anything (its expansion is not introducing a binding for list), the ordinary gensym approach is not applicable.

THe problem persists into a Lisp-2 like Common Lisp, in this form:

  (flet ((list (...) ....)) ;; binding in function namespace
     (mac ...))             ;; wants to use (list ...)

Since local functions are rarer than local variables, and extra care can be taken in how hey are named, Lisp-2 mitigates the problem. Common Lisp goes a step further and makes it undefined behavior if code redefines a standard function (lexically or otherwise).

Hygienic macros solve this aspect of the issue as well. A hygienic (mac ...) can use (list ...) in its expansion. The hygiene mechanism ensures that this calls that list which is lexically visible at the macro definition (probably the global one in the library) even if the target site shadows list with its own definition.

[patrec]

Again this is not fundamentally tied to whether functions and variables share a namespace, in clojure they do and yet your example above does not cause a problem. Watch:

Let's start out with a function:

   user=> (defn enlist [x] (if (list? x) x (list x)))
   #'user/enlist
   user=> (let [list 1] (enlist list))
   (1)

Create a (pointless) equivalent macro:

   user=> (defmacro enlist' [x] `(let [y# ~x] (if (list? y#) y# (list y#))))   
   user=> (let [list 1] (enlist' list))
   (1)

No hygiene and yet it works!

How? The secret is that backquote is automatically namespace qualifying:

    user=> '(list 1 2 3)
    (list 1 2 3)
    user=> `(list 1 2 3)
    (clojure.core/list 1 2 3)

In practice this + convenient gensym syntax (also demonstrated above) seems to be enough, just like in practice Common Lisp's approach of separate value and function cells for symbols + prohibition of rebinding function cells of symbols in the standard COMMON-LISP namespace is. Scheme has no cl style namespaces and no prohibition on redefining standard bindings and thus came up with a much more complex macro approach, at the gain of stronger hygiene guarantees (but see article above!) and the cost of an ugly design of considerable complexity that has a completely different sublanguage (non-orthogonally) baked in.

As far as purely hygiene is concerned that's IMO not a good trade-off at all; I could rattle off a long list of major usability gripes with both cl and clojure but problems caused by lack of macro hygiene wouldn't be on it.

Racket, which is basically several iterations beyond R*RS macros, OTOH is interesting because it's more principled approach gives you something qualitatively different to what you can do with clojure (no read-syntax control; bad error messages) and common lisp (readtables suck and break tooling; bad error messages).

[kazinator]

I'd have to say that a ham-fisted read-time solution which throws important backquote identities under the bus is worse than just coding carefully under the threat of unhygienic macros, and worse than the complexity of hygienic macros.

[patrec]

What concrete problem do you see with it? [edit: BTW you can quite easily implement common-lisp style backquote in clojure as a macro, because just like scheme clojure does not share common lisp's defect of baking unquoting into the readtable dispatch of backquote; i.e. unquotes outside of backquotes are syntactically valid; '~x -> (unquote x)]