[have-a-break]

Back prop and attention are just extensions of interpolation.

[rob_c]

By that logic it's all "just linear maths".

Back prop requires and limits to analytically differentiable in a normal way.

Attention is... Oh dear comparing linear regression to attention is comparing a diesel jet engine to a horse.

[aleph_naught]

It's all just a series of S(S(S(....S(0)))) anyways.