[kordlessagain]

Putting authentication on the site would be easier.

There a rub here, in that people expect to search things without being logged in. But then if you don't log in people, anyone can come calling, including bots. This then causes you to do things like get a third party to filter the data, which then affects the users by having to reroute their traffic to someone else to get rid of some of the visits you don't want from the bots.

And round and round.

Simple authentication to the site with tokens might solve the problem. If an IP comes calling that does so with out authentication, or payment, then hang the connection.