[simonw]

> I mention the problem of ‘hallucinations’ – when an AI model presents false or fabricated information as factual – and the need for a human face in court. The Sandie Peggie judgment allegedly contained AI-made errors. He waves this all away. ‘Temporary bugs and sentimental preferences. The economic argument is overwhelming.’

As usual with "AI replacing humans", the key thing to consider here is accountability.

I want to get my legal advice from someone who is accountable for that advice, and is willing to stake their professional reputation on that advice being correct.

An LLM can never be accountable.

I don't want an LLM for a lawyer. I want a lawyer armed with LLMs, who's more effective than the previous generations of lawyers.

(I'd also like them to be cheaper because they take less time to solve my problems, but I would hope that means they can take on more clients and maintain a healthy income that way even as each client takes less time.)

The closing paragraph of that story:

> ‘My niece is a lovely girl, really smart, great at school, and the other day she told me she wants to be a lawyer. And I thought, “Oh my God, my little niece wants to be a lawyer”, and I flat out told her. I said please do not destroy your life. Do not get into a lifetime of debt for a job that won’t exist in ten years. Or less.’

Uh oh. Here we go again, with the "don't bother studying computer science, it's 2002, all the jobs will be outsourced to cheaper countries in the next few years!". So glad I didn't listen to that advice back then!

[arctic-true]

When I was thinking about law school the big panic was about e-discovery tools: we wouldn’t need many lawyers anymore since we didn’t need to rifle through boxes of physical paper anymore! What happened instead was that, with the burden of collecting documents significantly reduced, we were able to start looking for needles in much bigger haystacks.

[atmavatar]

> I want a lawyer armed with LLMs, who's more effective than the previous generations of lawyers.

From what we've seen thus far, there's a non-zero chance the lawyer armed with LLMs will submit a brief generated by said LLM without reviewing it, which makes the judge none too happy.

Look at how people handle bringing their cell phones with them while driving. Some people won't use it at all. Some will play music (unrelated to driving but overall neutral as long as they aren't fiddling with it). Some will use it for GPS driving assistance (net positive). But, many will irresponsibly use it for texting/talking while driving, which is at least as bad as being inebriated and can lead to harming themselves and others.

Don't expect people to be any more or less responsible with LLMs.

[simonw]

I want my lawyer armed with LLMs to not do that.

There are some promising AI-driven tools these days that use search against archives of cases to help check that citations aren't garbage. I'm hoping lawyers start using them to help pick apart each other's laziness.

[danaris]

> I want my lawyer armed with LLMs to not do that.

The only way to guarantee that is to have a lawyer not armed with LLMs.

We've seen dozens of examples already of lawyers doing exactly that. (Some of them have then doubled down in court, to their eventual detriment.)

If you're making a habit of using LLMs to draft briefs for you, how long before you just forget to check the cited cases to replace the hallucinated ones with real ones? Or decide not to check, because surely they'll be fine this time...only they're not?

[simonw]

I believe it's possible for a lawyer to be more competent that, even if they have access to LLM tools.

[echelon]

> An LLM can never be accountable.

That's where AI businesses will make bank.

When they actually underwrite the risk of their models and sell that to clients - that's going to command an extremely high price premium.

The models aren't there yet, though.