[lostmsu]

> It takes an LLM millions of examples to learn what a human can in a couple of minutes

LLMs learn more than humans learn in a lifetime in under 2 years. I don't know why people keep repeating this "couple of minutes". Humans win on neither the data volume to learn something nor the time.

How much time do you need to learn lyrics of a song? How much time do you think a LLaMA 3.1 8B on a 2x3090 need? What if you need to remember it tomorrow?

[aithrowawaycomm]

They mean learning concepts, not rote factual information. I also hate this misanthropic “LLMs know more than average humans” falsehood. What it actually means “LLMs know more general purpose trivial than average humans” because average humans are busy learning things like what their boss is like, how their kids are doing in school, how precisely their car handles, etc.

[lostmsu]

Do you think the information in "what your boss is like" and "how your kids do in school" larger than amount of data you'd need to learn in order to give descent law advice on a spot?

Car handling is a bit harder to measure, precisely because LLMs aren't running cars quite yet, but also I am not aware of any experimental data saying they can't. So as far as I'm concerned nobody just tried that with LLMs of >70GB.

[pedrosorio]

> amount of data you'd need to learn in order to give descent law advice on a spot?

amount of data you'd need to learn to generate and cite fake court cases and give advice that may or not be correct with equal apparent confidence in both cases

fixed that for you

[lostmsu]

I could conceed the first point, in limited circumstances, but the second is moot to say the least.

Tool using big LLMs when asked can double-check their shit just like "real" lawyers.

As the confidence of advice, how much the rates of the mistakes are different between human lawyers and the latest GPT?

[pedrosorio]

> As the confidence of advice, how much the rates of the mistakes are different between human lawyers and the latest GPT?

Notice I am not talking about "rates of mistakes" (i.e. accuracy). I am talking about how confident they are depending on whether they know something.

It's a fair point that unfortunately many humans sound just as confident regardless of their knowledge, but "good" experts (lawyers or otherwise) are capable of saying "I don't know (let me check)", a feature LLMs still struggle with.

[lostmsu]

> I am talking about how confident they are depending on whether they know something.

IMHO, that's irrelevant. People don't really know they level of confidence either.

> feature LLMs still struggle with.

Even small LLMs are capable of doing that decently.

[someothherguyy]

> How much time do you need to learn lyrics of a song? How much time do you think a LLaMA 3.1 8B on a 2x3090 need?

Probably not the best example. How long does it take to input song lyrics into a file to have an operating system "learn" it?

[lostmsu]

Well, that just shows that the metric of learning time is clearly flawed. Although one could argue LLaMA learns while OS just writes info down as is.

But even the sibling concept comment is wrong, because it takes 4 years for _most_ people who are even capable of programming to learn programming, and current LLMs all took much less than that.