[jgrahamc]

"The problem used to be much worse, but then PG did this transformation manually for the important subset of the site (most importantly for links to the comments section -- these links no longer expire)."

I don't think this is strictly true. The comments and reply links simply link to a normal (non-closure) URL on the site and the page is generated from that URL in the normal manner.

[jules]

I don't understand the disagreement...isn't that the same thing I said? IIRC it used to be the case that all links could expire, then he did this transformation from closures stored on the server to closures manually serialized in the URLs (which results in a normal URL as you say). What I'm describing is how a web framework could do that automatically, which means you get the behavior and expressiveness of closures without the link expired issue. Though I can see that the explanation isn't very clear, but I don't know how to make it more clear. Edit: don't downvote him people, it is a perfectly fine comment.

[irahul]

> then he did this transformation from closures stored on the server to closures manually serialized in the URLs (which results in a normal URL as you say).

The reply links have a parameter 'id' - the id of the comment to which the reply is being posted. I would guess it's just passed to a normal function which adds the reply to the post.

May be you mean the same thing, but "serializing closures in the url"(where? all I see is the id of the post being replied to) isn't the same as params in the url which are then passed to a function.

[jules]

Yes, my point is that it is the same thing. What's happening is just manual closure conversion (http://en.wikipedia.org/wiki/Lambda_lifting), except the resulting data gets saved in the URL rather than the server. On the server, closures saved in the hash table, also have "params which are then passed to a function", except that the params (i.e. the free variables) are saved on the server inside the closure rather than in the URL.

If you have a lambda expression like `(lambda () (show-list-of-posts (+ 1 page-number)))`, the Scheme implementation does these things:

1. Introduce a global function definition with the same body as the lambda expression but with an extra parameter for the free variables. The free variables in the body of the function get replaced with expressions that extract their value from the extra parameter that has the free variables.

2. Convert the lambda expression to an expression that builds a pair of a function pointer to that global function, plus the values of the free variables.

For example the code:

    (define (show-list-of-posts page-number)
       ... display the rest of the homepage ...
       (link "More" (lambda () (show-list-of-posts (+ 1 page-number))))

Will get converted to something like this:

    ;; this is the extra global function that has the body of the lambda expression
    ;; note that the reference to `page-number` got 
    ;; replaced by `(extract-value "page-number" params)`
    (define (closure-324 params)
       (show-list-of-posts (+ 1 (extract-value "page-number" params))))

    ;; note that the lambda expression gets replaced by a create-closure expression
    (define (show-list-of-posts page-number)
       ... display the rest of the homepage ...
       (link "More" (create-closure closure-324 "page-number" page-number)))

create-closure creates a closure data structure where the first argument is the function pointer, and the rest of the arguments are the free variables.

Now, what happens if you want to remove this closure business in a web application, and instead use normal URLs?

First, you introduce a global request handler for the body of the lambda:

    (define-handler (post-list-handler params)
       (show-list-of-posts (+ 1 (extract-value "page-number" params))))

This would define the handler for news.ycombinator.com/post-list-handler?page-number=12.

Then, instead of the (link ...) with a closure, you just link to that url in the show-list-of-posts function:

    (define (show-list-of-posts page-number)
       ... display the rest of the homepage ...
       (link "More" (create-url post-list-handler "page-number" page-number)))

Compare these code snippets to the one above. Do you see the similarity to closure conversion? In both cases we:

1. Introduce a global function/handler for the body of the lambda.

2. That function/handler gets a `params` argument that has the free variables.

3. Everywhere a free variable is referenced in the body, it gets replaced by an expression that extracts the value from the params argument.

4. In place of the lambda expression, we have respectively a (create-closure func free-vars...) or a (create-url handler free-vars...)

So it's really completely analogous. That's why I say that we are just serializing the closure here, and this could be done automatically. Hopefully this makes it more clear what I mean, but maybe these details just make it less clear if you're not familiar with how closures are implemented (closure conversion)...