[Mikealcl]

You could do this without effecting normal traffic depending on uniqueness of ip doing the scraping.

Love the idea.

[swsieber]

I think you missed the point - if people show up at $PROXY expect nice stuff but see junk, then they won't move over to $REAL and instead blame $REAL.

E.g. you'd like some way to redirect people from $PROXY site to $REAL site, and disgusting content on $PROXY won't do that - it'll reflect poorly on $REAL

[heelix]

If you can identify the crawler - you can provide 'dynamic' content for that specific user context.

[nomel]

It's a proxy, so there's no "crawler". It's just an agent relaying to the user. Passing something to this proxy agent just passes it directly to the user.