[saurik]

There seemed to be some confusion during the question and answer segment regarding the relative hygiene of macros in Clojure near the end of the motivation and design talk; while it was totally off-topic for the video (and it thereby made sense to take it offline), I personally wish I had been around afterwards to ask the guy who seemed so adamant that syntax-case was fundamentally better than the Clojure solution (which he claimed didn't do it correctly) why that was the case.

I'm totally willing to believe it, but based on my understanding (which sadly is somewhat limited for Scheme, but fairly in-depth for Clojure) it isn't intuitive to me: it would seem like the way you escape hygiene in Clojure (which by default achieves correct hygiene by attaching namespaces to symbols read for macros or inside of quasi-quote) is quite similar in semantics--but simpler in practice due to being exceedingly less verbose--than using syntax->datum and datum->syntax.

[dherman]

That was me. :) Caveat: I know more about hygienic macros than I do about Clojure, so I'm not in a position to critique Clojure specifically.

Hygiene is (roughly speaking) about getting scope right by default but has never been about forcing it on the programmer. Moreover, there are two components to it, only one of which is easy to achieve in an unhygienic system. It's easy to ensure your macro renames introduced /bindings/ by using gensym. But if your macro introduces /references/ to existing variables, it's very hard to protect against those references getting captured at the site where clients call your macro.

I believe in Clojure they get around this for some cases by letting you fully qualify a reference to a library binding, for example. But what if your macro wants to refer to a variable that's local to it? Such as an unexported library function, or simply a local variable. Again, I don't know if Clojure has an answer to this.

One concrete example: write a `define-inline` macro-defining-macro. At the call site, a user might write

    (define some-local-variable /* something */)
    (define-inline (foo x)
      (+ x some-local-variable))

In an unhygienic system, they should first of all fully-qualify the `+` to be safe (yuck!) whereas a hygienic system just gets that right by default. But more critically, how can they be sure that some client of `foo` won't write:

    (define some-local-variable "something else")
    (foo 42)

Not only will that break, it's not clear how to fix it. A hygienic system gets this right.

I do agree that the state of the art in hygienic macros is too complex. I just haven't seen another system that makes this kind of thing work. But I would like to experiment with Clojure's macros more to see if they have an answer.

Dave

[saurik]

Clojure does not have these issues: when the macro is called, the symbols are already attributed with the full namespace qualification, and usage of quasi-quote inside of the macro definition will also apply namespace qualification to variables local to the definition of the macro; you have to go out of your way to break this. You should spend more time looking into it before claiming to people that it doesn't work correctly; you could easily have just said "that's a good question, we'll look into that after the talk" rather than telling the person that Clojure wasn't as good.

[dherman]

> Clojure does not have these issues: when the macro is called, the symbols are already attributed with the full namespace qualification, and usage of quasi-quote inside of the macro definition will also apply namespace qualification to variables local to the definition of the macro; you have to go out of your way to break this.

Again, not a Clojure expert, but a namespace is coarser-grained than individual local scopes, right? The problem I'm talking about is when you have a local variable inside a nested scope (e.g. inside a `let`). If this is not named by a namespace, then you would still get collisions.

Regardless, Clojure's approach seems to be much closer in spirit to a hygienic macro system: it attempts to get scoping correct by default, and allows you to intentionally capture.

> You should spend more time looking into it before claiming to people that it doesn't work correctly; you could easily have just said "that's a good question, we'll look into that after the talk" rather than telling the person that Clojure wasn't as good.

Fair enough as far as it goes. I did react snappily, but you didn't hear the offline conversation (this whole thing was a dialog at my office with a friend and colleague, incidentally) where I said "I'm not entirely sure, but I believe there are things you just can't express with systems like Clojure's." And we concluded, just as you reprimanded me to do, that we should look into it further when we have time.

Dave

[saurik]

Correct: I did not hear the private conversation. I only heard the talk that was made public along with this project that was posted here, and which was recommended as an information source, and pretty much serves as the web page and documentation for this project ;P.

> Again, not a Clojure expert, but a namespace is coarser-grained than individual local scopes, right? The problem I'm talking about is when you have a local variable inside a nested scope (e.g. inside a `let`). If this is not named by a namespace, then you would still get collisions.

Ok, so are you are concerned with the case where the person defining the macro uses a symbol from the namespace of the person using the macro but that symbol has been rebound by the user inside of a let surrounding the aforementioned usage of the macro?

If so, that requires a cyclic module dependency, which isn't allowed (as the namespace from which you are getting the symbol would need to be required, but it would have to require back to get access to the macro: it does eager name binding, so that can't happen).

If not, and you are just talking about the simpler and more obvious case of a let shadowing a binding inside of a larger scope used by a macro, that works fine. The following code prints "1100", despite the macro expanding to multiple uses of the same symbol "t".

    (def t 1000) 
    (defmacro run [x] 
        `(+ ~x t))
    (let [t 100] 
        (prn (run t)))

You might then wonder (as I have) whether this is implemented by simply renaming the variables bound by let to something random: that would be sufficient to implement this. However, if I go out of my way to unquote an unqualified symbol, I can capture: the following code prints "1200".

    (def t 1000)
    (defmacro run [x]
        `(+ ~x t ~'t))
    (let [t 100]
        (prn (run t)))

[dherman]

> Ok, so are you are concerned with the case where the person defining the macro uses a symbol from the namespace of the person using the macro but that symbol has been rebound by the user inside of a let surrounding the aforementioned usage of the macro?

> If so, that requires a cyclic module dependency...

Not necessarily. For example (forgive the Scheme syntax), all in one module:

    (define thing "outer thing")
    ;; define-inline is the above macro-defining-macro
    (define-inline (foo prefix)
      (string-append prefix thing))
    (let ((thing "inner thing"))
      (foo "should say outer thing: "))

[saurik]

That seems to be my "if not," case, which I provided some examples for; if this is different, can you please be more explicit? It seems like your "thing" is my "t" and your "foo" is my "run": the only difference is then that I went out of my way to make it more complex my passing the inner thing through the macro to demonstrate it wouldn't get mangled.

    (def thing "outer thing")
    (defmacro foo [prefix]
        `(str ~prefix thing))
    (let [thing "inner thing"]
        (prn (foo "should say outer thing: ")))

"should say outer thing: outer thing"

(edit:) Alternatively, maybe you are focussing on the define-inline "macro-defining macro"; you mentioned it here again as "the above", and you had used it above, but as it wasn't defined it didn't seem important. I tried to go ahead and implement it, although to be honest I feel like I did it wrong (spending more time thinking about it, I believe it is correct, modulo your definition of "inline"); that said, it "worked".

    (defmacro def-inline [[name & args] code]
        `(defmacro ~name ~(apply vector args)
            ~code))

    (def thing "outer thing")
    (def-inline [foo prefix]
        (str prefix thing))
    (let [thing "inner thing"]
        (prn (foo "should say outer thing: ")))

"should say outer thing: outer thing"