[sneak]

I think that in a strict interpretation of current copyright attitudes, web spidering/mirroring in general is probably an infringement if you then republish any part of that content in any way for your own benefit. Imagine if you made a "Google for Twitter" today; you'd be sued out of existence. I believe precisely this happened with LinkedIn, IIRC.

I think the concept of copyrights is fundamentally incompatible with an open web, ultimately. It should be that a webserver sending you a response to your request is an implicit usage license. After all, the webserver doesn't have to send you anything. If it does, we take that to be an implicit agreement that you can have and use the data. That should be codified as law, because it is the current convention.