|
|
|
|
|
|
by CodeBytes
148 days ago
|
|
|
It likely is AI scrapers essentially doing a DDoS. They use separate IPs (and vary the UA) to prevent blocking. I have a site which is currently being hit (over 10k requests today) and it looks like scrapers as every URL is different. If it was a DDoS, they would target costly pages like my search not every single URL. SQLite had the same thing: https://sqlite.org/forum/forumpost/7d3eb059f81ff694
As have a few other open source repositories. It looks like badly written crawlers trying to crawl sites as fast as possible. |
|
|