[vsgxvhdxh]

So all your head requests get filtered out when the db gets queried. Leaving all your gets and posts etc.

Also better hope your crawler doesn't hit any illegal sites (there are loads according to UK law).

[DanBC]

> (there are loads according to UK law).

No, there really aren't.