Hacker News new | ask | show | jobs
by pedrosorio 543 days ago
> amount of data you'd need to learn in order to give descent law advice on a spot?

amount of data you'd need to learn to generate and cite fake court cases and give advice that may or not be correct with equal apparent confidence in both cases

fixed that for you
1 comments
lostmsu 543 days ago
I could conceed the first point, in limited circumstances, but the second is moot to say the least.

Tool using big LLMs when asked can double-check their shit just like "real" lawyers.

As the confidence of advice, how much the rates of the mistakes are different between human lawyers and the latest GPT?

link
pedrosorio 543 days ago
> As the confidence of advice, how much the rates of the mistakes are different between human lawyers and the latest GPT?

Notice I am not talking about "rates of mistakes" (i.e. accuracy). I am talking about how confident they are depending on whether they know something.

It's a fair point that unfortunately many humans sound just as confident regardless of their knowledge, but "good" experts (lawyers or otherwise) are capable of saying "I don't know (let me check)", a feature LLMs still struggle with.

link
lostmsu 543 days ago
> I am talking about how confident they are depending on whether they know something.

IMHO, that's irrelevant. People don't really know they level of confidence either.

> feature LLMs still struggle with.

Even small LLMs are capable of doing that decently.

link