[lostmsu]

Do you think the information in "what your boss is like" and "how your kids do in school" larger than amount of data you'd need to learn in order to give descent law advice on a spot?

Car handling is a bit harder to measure, precisely because LLMs aren't running cars quite yet, but also I am not aware of any experimental data saying they can't. So as far as I'm concerned nobody just tried that with LLMs of >70GB.

[pedrosorio]

> amount of data you'd need to learn in order to give descent law advice on a spot?

amount of data you'd need to learn to generate and cite fake court cases and give advice that may or not be correct with equal apparent confidence in both cases

fixed that for you

[lostmsu]

I could conceed the first point, in limited circumstances, but the second is moot to say the least.

Tool using big LLMs when asked can double-check their shit just like "real" lawyers.

As the confidence of advice, how much the rates of the mistakes are different between human lawyers and the latest GPT?

[pedrosorio]

> As the confidence of advice, how much the rates of the mistakes are different between human lawyers and the latest GPT?

Notice I am not talking about "rates of mistakes" (i.e. accuracy). I am talking about how confident they are depending on whether they know something.

It's a fair point that unfortunately many humans sound just as confident regardless of their knowledge, but "good" experts (lawyers or otherwise) are capable of saying "I don't know (let me check)", a feature LLMs still struggle with.

[lostmsu]

> I am talking about how confident they are depending on whether they know something.

IMHO, that's irrelevant. People don't really know they level of confidence either.

> feature LLMs still struggle with.

Even small LLMs are capable of doing that decently.