Hacker News new | ask | show | jobs
by bwoj 479 days ago
On one level, yes you’re right. Computing weights and propagating values through an ANN is well defined and very algorithmic.

On the level where the learning is done and knowledge is represented in these networks there is no evidence anyone really understands how it works.

I suspect maybe at that level you can think of it as an algorithm with unreliable outputs. I don’t know what that idea gains over thinking it’s not algorithmic and just a heuristic approximation.

3 comments

"Heuristic" and "algorithmic" are not antipodes. A heuristic is a category of algorithm, specifically one that returns an approximate or probabilistic result. An example of a widely recognized algorithm that is also a heuristic is the Miller-Rabin primality test.

https://xlinux.nist.gov/dads/HTML/heuristic.html

https://xlinux.nist.gov/dads/HTML/millerRabin.html

https://en.wikipedia.org/wiki/Miller%E2%80%93Rabin_primality...

“Algorithm” just means something which follows a series of steps (like a recipe). It absolutely does not require understanding and doesn’t require determinism or reliable outputs. I am sympathetic to the distinction that (I think) you’re trying to make but ANNs and inference are most certainly algorithms.
> On the level where the learning is done and knowledge is represented in these networks there is no evidence anyone really understands how it works.

It is hard to assess the comment above. Depending on what you mean, it is incorrect, inaccurate, and/or poorly framed.

The word “really” is a weasel word. It suggests there is some sort of threshold of understanding, but the threshold is not explained and is probably arbitrary. The problem with these kinds of statements is that they are very hard to pin down. They use a rhetorical technique that allows a person to move the goal posts repeatedly.

This line of discussion is well covered by critics of the word “emergence”.