[jMyles]

> the most private or traumatizing moments a vulnerable person has

...and in this hypothetical, this person is serving them via an unauthenticated http server and hoping that clients will respect robots.txt?

[bigbuppo]

Robots are supposed to behave. It was a solved problem 30 years ago until AI bros unsolved it. Any entity that does not obey robots.txt is by definition a malicious actor.