|
|
|
|
|
|
by input_sh
37 days ago
|
|
|
Yes, we know, its purpose is to guide the bots, not forcibly block them. That said, one of the biggest websites in the world not respecting it is definitely a noteworthy story. Hopefully another one of the biggest websites in the world (formerly known as Twitter) eventually respects it as well instead of not even disclosing itself via a user agent and pretending to be Safari running on iOS. |
|
|
You're talking about one (yes, biggest) but millions of other bots don't follow must be a bigger story.