[pooyamo]

Does any expert in the field know whether it is really the case that this intelligence we are seeing with frontier models is an "emerging" phenomena, only coming up when the architecture is scaled?

Like isn't it weird that the 1 million parameter model with the same architecture can't solve basic puzzles but suddenly the 1 trillion parameter can conjure up counter-examples for the Jacobian conjecture?

It's unintuitive since, to the best of my knowledge, one of the basic tenants of algorithm development was that you can't just brute-force your way towards a solution for some complex problems, e.g. naive sorting algorithms suddenly won't beat quicksort if you put more processing to them, but in the modern LLM scene it seems people are in a race to scaling up, experimenting empirically and hoping the same algorithm/architecture comes to a solution.

[jlamberts]

This is actually a well-known phenomenon in ML, called "The Bitter Lesson".

> One thing that should be learned from the bitter lesson is the great power of general purpose methods, of methods that continue to scale with increased computation even as the available computation becomes very great. The two methods that seem to scale arbitrarily in this way are search and learning.

The full essay is worth a read, it's pretty short http://www.incompleteideas.net/IncIdeas/BitterLesson.html

[BinaryRage]

Welch Labs series on this is great too:

https://youtu.be/2hcsmtkSzIw

[hnfong]

> one of the basic tenants of algorithm development was that you can't just brute-force your way towards a solution for some complex problems

It's kind of sad that popular CS textbooks often focus on solving precise problems with lowest theoretical complexity bounds while ignoring more practical (but generally applicable) computation techniques.

In machine learning they call it "gradient descent", which in older days had analogies in techniques called "hill climbing", "local search" and "simulated annealing". Basically you have a function you need to optimize for, and you clumsily tweak the parameters so that you get the (locally) max/min value you wanted. These techniques were great at finding approximate, locally maximal solutions without trying all the possibilities at once (which is more akin to the kind of "brute force" in the traditional CS context).

I guess because these techniques were generally applicable yet the outputs were approximate and you couldn't analyze them much (no fancy O(n log n)), the theorists did not find them interesting and thus were not put into the spotlight of student's learning curricula.

In modern machine learning they do this gradient descent thing which is also tweaking the parameters bit by bit to optimize for the loss function, except that the parameters are now in the billions and trillions. The compute required is huge of course, but it's actually quite an "efficient" process, and it's not actually doing much of "brute forcing" at all. During training, the process is essentially, almost equivalent to, compressing the many many trillions of tokens of training data. To me it's quite amazing that they manage to complete such a process within a couple months of training, even if they have hundreds of thousands of GPUs...

[nuancebydefault]

In my math syllabus for Engineering there was a book "numeric analysis", it showed how you could find the solution to weird equations like x=ln(x)

I thought this is nowadays called gradient descent

[IanCal]

It might be that what we consider a basic and very hard puzzle are extremely close together on a more absolute scale. The difference is often for us what proportion of humans can solve it. And the low end of that is still quite high up - animals that can solve things that are very basic for the vast majority of humans are pretty rare and known about, yet are capable of quite complex actions and learning and aren’t wildly different in scale of neurons to us.

Going from 1m to 1T params is also a scaling of a million times. It’s like going from a human brain down to one percent in size in each direction or just a few mm.

[joefourier]

> Like isn't it weird that the 1 million parameter model with the same architecture can't solve basic puzzles but suddenly the 1 trillion parameter can conjure up counter-examples for the Jacobian conjecture?

I'm not sure what you mean? You can see the intelligence of LLMs progress predictably and stably according to scaling laws. LLMs have to encode language in addition to intelligence so there's a minimum bound for them to output sensible text (you can train specialised tiny models to solve basic puzzles without language). Start at around 127M and compare models of increasing parameters and you'll see a clear progression in intelligence.

> It's unintuitive since, to the best of my knowledge, one of the basic tenants of algorithm development was that you can't just brute-force your way towards a solution for some complex problems, e.g. naive sorting algorithms suddenly won't beat quicksort if you put more processing to them

How is that a basic tenet? Simple, easier to parallelise algorithms that have lower memory requirements, or can take better advantage of hardware, or don't hit a plateau the more compute you throw at them, can absolutely beat cleverer algorithms. E.g. brute forcing rendering with Monte Carlo path tracing will give you more physically accurate results than ray tracing or rasterisation algorithms that rely on a bundle of hacks to approximate global illumination, transparency smooth shading, etc.

[sota_pop]

It is _emerging_ in the sense that complex (sic non-linear) systems can exhibit unintended or unexpected behaviors.

It is _intelligent_ in the sense that it optimizes a thing that is hard for humans to not anthropomorphize.

Things like percolation theory, swarm theory, and others are similar topics in “complex systems”. Neural networks are interesting because they combine aspects of both complex systems and dynamical/adaptive systems (ie systems with a feedback loop).

A neural network at its very core is a function fitting algorithm. It stores matrices of parameters (ie weights and biases) such that every parameter (and combinations thereof) captures a relationship of your data in exactly the same way the slope and intercept are obtained through Linear Interpolation. All of the Regularization tricks applied just are attempts to incentivize a given parameter to not encode any trends too specifically (think an instance vs a type).

In this way you can ask yourself “how many aspects of your data are required to capture it adequately?” This is what scaling offers.

[pornel]

IANAMLE, but there is "grokking" that makes models learn to actually generalize, even after you give them enough parameters that would let them memorize the dataset:

https://en.wikipedia.org/wiki/Grokking_(machine_learning)

High-dimensional gradient descent behaves very differently than the simplified 3d visualisations we use to demonstrate it, and has lots of ways out of local minima:

https://www.youtube.com/watch?v=NrO20Jb-hy0

so it seems like there is a benefit to giving models more space to learn in rather than forcing them to compress the knowledge from the start.

[CamperBob2]

Exactly, I was going to suggest the Welch Labs videos on grokking. Especially the newer one at https://youtu.be/D8GOeCFFby4?si=yLI9zzcjsnEELUqy . They are really well done and really eye-opening.

[neerajk]

There is a Sanjeev Arora paper "A Theory for Emergence of Complex Skills in Language Models" (https://arxiv.org/pdf/2307.15936) on this subject. The key idea is there is cross entropy (how "surprised" the model is with the "correct" next token, lower is better), some of which is inherent in the language and therefore unavoidable, and the rest is model error, and that this portion of the cross entropy is reduced with scaling.

And as scaling reduces a model's excess entropy, the model can become good at combinations of skills much faster than you would expect if it had to separately see and memorize every combination. They call this "slingshot generalization".

[coderenegade]

I'm out of the scene these days, and my field is more RL than LLMs, but my take is that all of this is telling us that meaning (and intelligence) is in the medium. If an LLM is a function mapping an input to some stateful internal representation that in-turn then gets mapped to an output, a smaller model probably doesn't have the capacity to learn that internal mapping, at least not from scratch. Or it would take a very long time, in same way that a naive sorting algorithm would still get there, but it might be computationally prohibitive in practice.

The bigger model has a better chance of gaining a foothold in that internal representation space where inputs are mapped to meanings and outputs, and eventually, it optimizes to the point where most of the weights aren't doing much. It's not immediately clear how much expressivity is required by the network to learn that space, but so far the answer seems to be in the billions of parameters.

The more interesting question to me is to what extent we should expect the models to be invariant to data. For instance, if I learn a certain type of analysis, that skill shouldn't depend on the data I'm looking at-- it should be repeatable for any given data of the same type/class. I'm curious to what extent skills are embedded in the weights versus data and "facts". My hunch for why mathematical reasoning and programming resulted in large step changes in model performance across the board is because these are inherently skills that are widely repeatable for a large class of tasks. The ability to express programmatic logic is invariant to both the language and the task at hand. And to me, that's how you get to smaller models: by focusing on the skills.

[hodgehog11]

My expertise lies in deep learning theory, and yes, the "intelligence" is coming primarily from scaling up, among other things. There are good reasons for this, but essentially it comes down to taking advantage of a narrow statistical trick, where a very well-crafted model/optimizer pair that has a strong implicit bias toward simplicity can exhibit progressively increasing performance with respect to model size. Marcus Hutter's lab has shown that you can phrase this in terms of Solomonoff induction, so this bias is truly universally effective. An effective bias can continue to improve performance with larger model sizes by taking advantage of the curse of dimensionality in a way not dissimilar to how more data generally gives you a better answer (indeed, there is a duality taking place here, but I digress).

To be clear, it is an extremely narrow model class that can do this; we just got "lucky" and worked our way to it. That's why we still teach general statistical principles which often forbid this sort of behavior as a rule of thumb.

[thomasahle]

Here's one way it could happen:

Let's say there's some circuit that does problem solving of the kind we call intelligence.

We dont know what this circuit looks like, but it exists in our brain.

Doing regression on outputs from the brain (e.g. internet text) with enough parameters, we can "fit" our model to this circuit.

But if you try to fit it with fewer parameters than it needs, you're just going to get some linear approximation.

[hendiatris]

So basically a Nyquist rate type of concept.

[esafak]

https://en.wikipedia.org/wiki/Information_bottleneck_method

[lacunary]

what is the approximation linear in?

[lern_too_spel]

In the output of this nonlinear model /s.

[chpatrick]

http://www.incompleteideas.net/IncIdeas/BitterLesson.html

[ACCount37]

Increasing raw model scale is one of the most consistent and reliable ways of increasing its intelligence.

In a way, training an AI is: using an algorithm to find, discover and refine other algorithms computationally. When we scale, we pour more raw inputs into finding the right algorithms for a given objective. Is it surprising, then, that we find a better algorithm for it?

As a very stupid analogy - for a given task, a small model's internal algorithm might, under its capacity and training signal constraints, top out at slightly above "bubble sort". While a larger one could dig deeper and get closer to "quicksort" internally. A naive, dirty algorithm got replaced by a more sophisticated algorithm that performs better.

Keep in mind: intelligence is very much not a binary. There's no "threshold" at which a model goes from "this is dumb statistics" to "this is actual intelligence". A 1B LLM and a 10T LLM both have some amount of intelligence. It's just that one would have intelligence that's so weak, underdeveloped, and overindexed on statistical regularities that it's very easy to dismiss it altogether. And the other might have enough of it to snipe unresolved conjectures with novel counterexamples in math. Makes it considerably harder to dismiss it outright. While the curve between the still two looks less like an abrupt jump and more like little increments building up to an avalanche.

The jump in something advanced and specific like "math abilities" can look quite sharp - but under it, there are far more generic capabilities that back it. They build up slowly to eventually enable that jump. A more advanced model makes less reasoning mistakes and recovers from reasoning mistakes more gracefully - two very generic capabilities - but once those capabilities improve enough, a whole new type of logic problem might fall to it.

[storus]

There was a presentation at CS 25 Transformers United about some phenomena like chain-of-thought emerging only after training LLMs with at least 1T tokens and only in LLMs of certain size.

https://youtu.be/tVtOevLrt5U?t=923

[3836293648]

Emergent phenomena = pass/fail grading of multi part problems. If you're just gonna count it as a fail you're not gonna see the progress until everything works, even if every single subproblem improves linearly.

[knaeckeKami]

Mice have around 70 Million Neurons, humans around 50-100 billion.

Yes, you can't compare biological neurons to parameters in an LLM 1:1, neurons do much more, but still - It is plausible that higher intelligence just requires scale.

At least not even evolution over millions of years seems to have found a way to produce human-level intelligence in a smaller scale, even though it had a lot of incentives to do so (the human brain needs a lot of kcal).

[Mr_Minderbinder]

A closer biological analogue to an LLM parameter would be a synapse. Estimates vary greatly on how many there are in the human brain, but they are generally on the order of ten to the 14th or 15th power (hundreds of trillions to quadrillions). Using this analogy it is clear that even frontier LLMs, or at least those with publicly disclosed parameter counts, are not yet at the same scale and complexity as the human brain.

[mohsen1]

> one of the basic tenants of algorithm development was that you can't just brute-force your way towards a solution for some complex problems

Mote-Carlo is pretty useful still. Not sure if your statement holds

[lacunary]

it's certainly not a definite procedure for determining if an arbitrary mathematical statement is true or not. it's more like educated guess and check which definitely scales up