[sota_pop]

It is _emerging_ in the sense that complex (sic non-linear) systems can exhibit unintended or unexpected behaviors.

It is _intelligent_ in the sense that it optimizes a thing that is hard for humans to not anthropomorphize.

Things like percolation theory, swarm theory, and others are similar topics in “complex systems”. Neural networks are interesting because they combine aspects of both complex systems and dynamical/adaptive systems (ie systems with a feedback loop).

A neural network at its very core is a function fitting algorithm. It stores matrices of parameters (ie weights and biases) such that every parameter (and combinations thereof) captures a relationship of your data in exactly the same way the slope and intercept are obtained through Linear Interpolation. All of the Regularization tricks applied just are attempts to incentivize a given parameter to not encode any trends too specifically (think an instance vs a type).

In this way you can ask yourself “how many aspects of your data are required to capture it adequately?” This is what scaling offers.