[Spooky23]

Worse, judges are writing decisions and evaluating cases based on this.

The problem is that we’re going to have people litigating court cases and stupid bureaucratic nonsense within companies by lobbing nonsense that nobody has read at each other. Everyone will wink and nod, but these organizations and institutions will be exponentially more clueless.

As technologists, it’s easy to shrug. What do you do when a judge takes your kid away after using an LLM to read and interpret bullshit generated case notes that a CPS worker generated with another LLM?

[jprete]

Technologists are spectacularly bad at understanding how much the system relies on trusting that people will do something vaguely reasonable/legal/prosocial, just how that trust is built up and preserved, and how fragile that trust really is. A lot of Big Tech disruption and moneymaking has been based on ignoring this point to consume systemic trust without replenishing it.

[namelessoracle]

This already happens to an extent though?

Replace LLM with "clerk" and "generated case notes" with "copy pasted from my standard legally approved phrasing" playbook. Government workers who fill out standard reports already have standard forms pre filled with the results they know they are landing out, with certain sections that differ ready to be edited while the rest stays the same, look at things like warrants as an example. When I worked a government job i was literally handed templates by my supervisor of "pre approved ways to phrase things", that they percieved would help avoid lawsuits or any contest.

[Spooky23]

The difference is that when you fuck up, you or your supervisor will be held accountable. Everything is fine until it isn’t.

With the LLM, everyone will clutch their pearls and be shocked.

[nonethewiser]

> The problem is that we’re going to have people litigating court cases and stupid bureaucratic nonsense within companies by lobbing nonsense that nobody has read at each other.

At an abstract level, this sounds like a precursor to war. Everyone has competing interests and the peaceful way to resolve them is communication. When that stops force starts.

Maybe the out is that there is some automated sense of listening and agreement.

[narrator]

I think judges will continue to enjoy sanctioning lawyers for quoting fake cases. It's pretty easy to check if the citations are real or not. The legal system is not some API you can just spam with bullshit. There are serious consequences for doing that. The fun part comes when the Woke API refuses to make arguments for case law that goes against its woke programming.

[vmfunction]

What you are describing has being happening for a long time in tech. It was suppose to make our life easier and do things we don't want to do, but has it? Seems like we are just become salves of digital technologies now days.

Instead what we got instead are surveillance capitalism and now this LLM/AI stuff that becomes much better are emulating human behaviour and languages. UX in software has NOT improved, and I dare to say it has gotten worse. Just feel the performance of iOS 6 on iPod touch vs the lates iOS. Tech over quarter century has just tech getting more complex, and made more easily to centralised and consolidate control. Is this what we really want?