[hooloovoo_zoo]

What if LinkedIn adds a visibility option in addition to public/private profile that says "I want LinkedIn to prevent robots from scraping my profile."? What if LinkedIn enables that mode by default? Can they then continue preventing scrapers?

[alt_f4]

I think they can, but they won't because robots includes search engines and blacklisting search engines from user profiles will very negatively impact their metrics.

[perl4ever]

You're implying they can't discriminate. So does this case make robots.txt illegal?

[alt_f4]

First off, robots.txt is optional. It's neither a technical nor a legal limitation at this point.

Second, OP's argument suggests a UI option that gives or removes user consent from all robots in general. Unless they plan to word it: "allow robots that we like that are good for us. but disallow other robots", I don't think it's okay to discriminate by either allowing/banning a particular robot, as that is not what the user agreed to.