Hacker News new | ask | show | jobs
by elias94 1410 days ago
> has been upwards of 15 queries per second from bots

What type of queries are they generating? For what purpose are querying Marginalia? Scraping and filling internal search engines?

> If anyone could go ahead and find a solution to this mess

I would maybe trying to investigate why are querying your search engine. Is for the search results? Maybe from there you can create and sell an API service. Is for the wiki? Is for research purpose?

I would love to see some data, raw or with some behavior derived from it.

1 comments

Most of the queries don't seem to be tailored toward my search engine, they're ridiculously over-specified and typically don't return any results at all.

As I've mentioned in another comment, my best guess is they're betting it's backed by google, and are attempting to poison their search term suggestions. The queries I've been getting are fairly long and highly specific, often within e-pharma or online casino or similarly sketchy areas.

Like

> cialis 50mg online pharmacy canada price

Either that, or nonsense like the below, where they appear to be looking for CMSes to exploit (although I don't understand the appendage at the end)

> "Please enter the email address associated with your User account. Your username will be emailed to the email address on file." Finestre Antirumore Torino

> affordable local seo services "Din epostadress delas eller publiceras aldrig Obligatoriska flt r markerade med"

> "You are not logged in. (Login)" Country "City/Town" "Web page" erst

Point is, none of these queries actually return anything at all. I don't offer real full text search, for one. And the queries are much too long.