Hacker News new | ask | show | jobs
by pzmarzly 604 days ago
There are some browser plugins that try to guess what technologies are used by the website you are visiting. I hope the better ones can guess it by just looking at HTML and HTTP headers, but wouldn't be surprised if others were querying some known endpoints.
1 comments

Then they are, by definition, bots scriping the site for informations, and should start by the robots.txt