Hacker News new | ask | show | jobs
by pierlu 2560 days ago
Modern crawlers can easily follow javascript or some basic POSTs. So, apart from the insane state-changing GET requests in the code, unless you are spidering your own application, I would not give a crawler admin credential. Anyway a very nice anecdote of the two-thousands decade. Never ever forget the basics.