[RussianCow]

> Blindly trusting these models without any verification will put innocent people in prison.

I don't think anybody is suggesting this. But if the models can gleam information/insights that humans can't, that's still valuable, even if it's wrong some percentage of the time.

[snowe2010]

This is what happened with dna testing at the beginning. Prosecutors claimed it was x percentage accurate when in fact it was hilariously inaccurate. People thought the data was valuable when it wasn’t.

[fkyoureadthedoc]

I don't see that as particularly analogous. The average person will have had LLM technology in their own hands for years, whereas with DNA it was completely foreign to them and their only choice really was to trust the experts. And on top of that DNA testing matured and is very useful now.