If I have a "no publicity" sign in my mailbox and you dump 500 lbs of flyers and magazines by my door every week for a month and cause me to lose money dealing with all the trash, I think I'd have a reasonable ground to sue even if there's no contract saying you need to respect my wish.
End of the day the claim is someone's action caused someone else undue financial burden in an way that is not easily prevented beforehand, so I wouldn't say it's a 100% clear case but I'm also not sure a judge wouldn't entertain it.
I didn't say no one could sue, anyone can sue anyone for anything if they have the time and the money. I said I didn't think someone could sue over non-compliance with robots.txt and win.
If it were possible, someone would have done it by now. It hasn't happened because robots.txt has absolutely no legal weight whatsoever. It's entirely voluntary, which means it's perfectly legal not to volunteer.
But if you or anyone else wants to waste their time tilting at legal windmills, have fun ¯\_(ツ)_/¯.
You don't even need to mention robots.txt, there's plenty of people that have been sued for crawling and had to stop it and pay damages, just lookup "crawling lawsuits".
Your verbs, “sue” and “win”, are separated by ~16 words of flowery language. It’s not surprising that people gave up partway through and reacted to just the first verb.
The „well everyone can sue anyone for anything“ is a non-serious gotcha answer anyways. If someone asks „can I sue XY because of YZ“, the always mean „and have a chance of winning“. Just suing someone without any chance of winning isn’t very interesting.
End of the day the claim is someone's action caused someone else undue financial burden in an way that is not easily prevented beforehand, so I wouldn't say it's a 100% clear case but I'm also not sure a judge wouldn't entertain it.