[jmalicki]

Performance enhancements are huge though.

If you can make the existing model faster, you can then save your inference budget to then make your model bigger, which then makes it smarter.

A lot of how smart the models can be comes down to budget. If you can make your existing thing cheaper, you can instead make it bigger for the same price.

[TheHalfDeafChef]

Not really “smarter” though? It’s just a big probability engine.

(Not trying to flame bait or anything. I just wouldn’t call LLM as exhibiting intelligence. It is great at making connections based on probability but doesn’t have a semantic understanding of what it is doing)

[stevenhuang]

You do realize modern neuroscience considers the human brain as "just" a probability engine and that intelligence may well be the ability for an organism to predict well.

> doesn’t have a semantic understanding of what it is doing

I hope you realize this is an area of open, active research.

[Chu4eeno]

Didn't neuroscience some big scandals about bad statistics and overstating their findings (in addition to normal issues like replication)? Look up at least the "dead salmon study" (hint: it's related to fMRI, and you can probably guess its conclusions from its nickname). The "Voodoo Correlations" and "Cluster Failure" papers are also a bit eye-opening.

In general we (humans) need to be humble about the limitations of our knowledge about how we function, it's an insanely complicated problem.

[jmalicki]

> In general we (humans) need to be humble about the limitations of our knowledge about how we function, it's an insanely complicated problem.

We do.

Which is why we shouldn't be assuming we're more than just probability engines, or be assuming we have more consciousness than a neural network.

[otabdeveloper4]

> to then make your model bigger, which then makes it smarter

There's diminishing returns and at some point making a model bigger makes it dumber.

[lobocinza]

Maybe due to lack of data and dimensions other than words.