[amelius]

Well, what's different is that humans invent new abstractions along the way such as complex numbers and Fourier transforms.

[gowld]

A neural network is nothing but a heap of new abstractions from data.

[MrPatan]

Not a lot of humans do

[skepticATX]

Every single human has abstractions that are unique to them. Your world model isn’t the same as mine.

It’s just that usually these abstractions are fuzzy and hard to formalize, so they aren’t shared. It doesn’t mean that they don’t exist.

[sdenton4]

This is laughable... Neural networks also have basically unknowable abstractions encoded in their weights. There was some work not long ago which taught an ANN to do modular arithmetic, and found that it was doing Fourier transforms with the learned weights....

[amelius]

When a human has done the same thing many times they tend to try to generalize and take shortcuts. And make tools. Perhaps I missed something but I haven't seen a neural net do that.

[mitthrowaway2]

Is that very different than the distillation and amplification process that happens during training? Where the neural net learns to predict in one step what initially required several steps of iterated execution.

[WalterSear]

IMHO, yes. It's not an (internal) invention occurring as a result of insight into the process - it's an (external) human training one model on the output of another model.

[littlestymaar]

True, but we're talking about Olympiad level math skills here.