| > That the provider's business needs necessitate the this behaviour... No, please don't move past this so quick, because I'm not convinced they have the need, and in some markets (like the US) it is a violation of civil rights, to show people different content based on their ethnicity[1] because those people might have a claim that supersedes anything they might have signed or clicked-through. That Anthropic did something they could obvious be sued for constitutional violations in multiple countries is shocking. > what else [are] they're harvesting from my machine? PII? Assume everything, and yet I think this is more insidious than mere exfiltration, and we should go further: The LLM can respond to these magic quote marks directly, which means it can be trained to give people bad/different advice without those markers being so visible to people using debugging tools. That's so unethical, the laws on this potentially so severe, Anthropic could be facing unlimited damages, from any one example combined with this article, which means either they have a really stupid management team, or were given a promise of legal immunity in some way. Neither of those things should be what you should want to base your next big idea on. [1]: For a simple example, showing an ad for (say) mortgage offers and targeting people by race/ethnicity/gender is totally illegal, but it's also illegal if you make a list of targeting criteria that just happen to select for a protected class. |