[nyrikki]

If the goal was to create an artificial neural network that better approximated the biological human brain, yes the perceptron model is insufficient.

If your goal is to produce a useful model on real hardware and it works...no

Remember the constraints of ANNs being universal approximaters (in theory)

1) The function you are learning needs to be continuous 2) Your model is over a closed, bounded subset of R^n 3) The activation function is bounded and monodial

Obviously that is the theoretical UAT constraints. For gradient decent typically used in real ML models, the constraint of finding only smooth approximations of continuous functions can be problematic depending on your needs.

But people leveraged phlogiston theory for beer brewing with great success and obviously Newtonian Mechanics is good enough for many tasks.

SNNs in theory should be able to solve problems that are challenging for perceptron models, but as I said, features like riddled basins are problematic so far.

https://arxiv.org/abs/1711.02160

[Jensson]

> 1) The function you are learning needs to be continuous

Seems like a bad limitation when you try to model reasoning based on facts and logic, there are many things there that are just true or false and no spectrum to it. There is no "kinda true" in those circumstances, you should only get 1 or 0 and never any value between.

[nyrikki]

Perceptrons are binary classifiers, that output 0 or 1, based on a threshold.

While not practical to find or use, any feed forward network supervised is effectively a paramedic linear regression.

Think of an Excel line graph, drawing lines between points, with the above the line being 'true', or when the soma fires.

That is how perceptrons work.

Single layer perceptrons cannot represent linearly inseparable functions like XOR or band pass.

A single biological neurons can use the timing of pulses, band pass, change the rate of pulses etc... before it ever reaches the soma.

Not all problem can be reduced to decision problems and not all of them can use constant depth threshold circuits, which hard attention is.

An LLM can be a very reliable threshold or majority gates as an example, but cannot generalize PARITY.

Basically statistical learning inherited the same limits of statistics.

"This statement is 'False'" is a good paradox to use as a lens.