[aaron-lebo]

The key to understanding macros is to be quite clear about the distinction between the code that generates code (macros) and the code that eventually makes up the program (everything else). When you write macros, you're writing programs that will be used by the compiler to generate the code that will then be compiled. Only after all the macros have been fully expanded and the resulting code compiled can the program actually be run. The time when macros run is called macro expansion time; this is distinct from runtime, when regular code, including the code generated by macros, runs.

http://www.gigamonkeys.com/book/macros-defining-your-own.htm...

[teddyh]

Is it not the fact that macros can be defined and, more importantly, redefined at runtime? If so, would that not mean that macros must also be expanded at runtime?

[lispm]

If you have compiled code, you have to recompile the code when you change a macro. It's not done automagically.

If you have interpreted Lisp code, then the code can use the new macro automagically.

[lispm]

This is the view of a compiled execution. Lisp can also be interpreted from source as data. Then macro-expansion time is interleaved with execution time. Each use of a macro form may trigger a macro expansion.

Remember, a Lisp interpreter interprets Lisp source as data. Not byte code. Unlike Python, Java, Smalltalk, ... which all have popular implementations which compile to byte code and execute that byte code in a byte code interpreter, aka virtual machine.

Let's use a Lisp interpreter, here from LispWorks:

We define a primitive MY-IF macro. It expands into a simple IF use. But the macro will also count the number of macro expansions.

    CL-USER 46 > (defparameter *myif-counter* 0)
    *MYIF-COUNTER*

    CL-USER 47 > (defmacro my-if (c a b)
                   (incf *myif-counter*)
                   `(if ,c ,a ,b))
    MY-IF

LispWorks can trace macros, too.

    CL-USER 48 > (trace my-if)
    (MY-IF)

Now a simple function which uses our macro:

    CL-USER 49 > (defun fac (n)
                   (my-if (= 1 n)
                          1
                          (* n (fac (1- n)))))
    FAC

Now we use it and we will see the trace information for the macro use: you see the incoming form and the result form.

    CL-USER 50 > (fac 2)
    0 MY-IF > ...
      >> COMPILER::FORM        : (MY-IF (= 1 N) 1 (* N (FAC (1- N))))
      >> COMPILER::ENVIRONMENT : #<Augmented Environment venv NIL fenv ((#:FUNCTOR-MARKER . #<COMPILER::FLET-INFO (# # # #)>)) benv NIL tenv NIL decl NIL>
    0 MY-IF < ...
      << VALUE-0 : (IF (= 1 N) 1 (* N (FAC (1- N))))
    0 MY-IF > ...
      >> COMPILER::FORM        : (MY-IF (= 1 N) 1 (* N (FAC (1- N))))
      >> COMPILER::ENVIRONMENT : #<Augmented Environment venv (#<Venv 275415194600  N>) fenv ((#:SOURCE-LEVEL-ENVIRONMENT-MARKER . #<COMPILER::FLET-INFO (NIL . #)>) (#:FUNCTOR-MARKER . #<COMPILER::FLET-INFO (# # # #)>)) benv NIL tenv NIL decl NIL>
    0 MY-IF < ...
      << VALUE-0 : (IF (= 1 N) 1 (* N (FAC (1- N))))
    0 MY-IF > ...
      >> COMPILER::FORM        : (MY-IF (= 1 N) 1 (* N (FAC (1- N))))
      >> COMPILER::ENVIRONMENT : #<Augmented Environment venv NIL fenv ((#:FUNCTOR-MARKER . #<COMPILER::FLET-INFO (# # # #)>)) benv NIL tenv NIL decl NIL>
    0 MY-IF < ...
      << VALUE-0 : (IF (= 1 N) 1 (* N (FAC (1- N))))
    0 MY-IF > ...
      >> COMPILER::FORM        : (MY-IF (= 1 N) 1 (* N (FAC (1- N))))
      >> COMPILER::ENVIRONMENT : #<Augmented Environment venv (#<Venv 275416002360  N>) fenv ((#:SOURCE-LEVEL-ENVIRONMENT-MARKER . #<COMPILER::FLET-INFO (NIL . #)>) (#:FUNCTOR-MARKER . #<COMPILER::FLET-INFO (# # # #)>)) benv NIL tenv NIL decl NIL>
    0 MY-IF < ...
      << VALUE-0 : (IF (= 1 N) 1 (* N (FAC (1- N))))
    2

Let's see how often our macro function has been used to expand code:

    CL-USER 51 > *myif-counter*
    4