Hacker News new | ask | show | jobs
by oefrha 742 days ago
I tried with a nonexistent path, here are the user agents I got:

  Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36
  Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36
It's apparently built to evade detection.
2 comments
menacingly 741 days ago
the more reasonable assumption is that it uses a third party crawl service.

but still, it doesn't matter, it's acting on behalf of a user, and you aren't entitled to know what software your users run. One consequence of putting stuff on the public internet is that it's like public.

link
moneywoes 742 days ago
what sort of evasion technique is being used here, random user agents on each attempt?
link
CaptainOfCoit 741 days ago
> what sort of evasion technique is being used here

Using a user-agent that looks like a desktop user-agent, rather than including the name of the actual project/product that is being used.

I understand why though, plenty of websites block anything that looks like non-desktop/mobile user-agents, so makes sense. Besides the pragmatic reason, I also agree with menacingly that it doesn't matter, people should be able to use whatever user-agent they want to access the content you've put publicly on the internet.

link