[vegesm]

I don't really get the first part of your comment. You say an NN is "just" compression + kNN and does no representational learning. But finding a compression (a transformation in other words) that makes kNN feasible on the data is exactly what people mean when they say it finds a hidden representation. It is a highly non-trivial task: e.g. simple distance in pixel space between images would get you useless results.

[mjburgess]

People have the notion that a latent representation in the animal sense, ie., a concept -- is the same thing as your "representation" in the NN sense.

That's not the case. You're right that if I find a predictive compressions of faces, say F1...n then they arent literal "rememberings". And they seem to be able to participate in a decision process (eg., classification) which doesnt seem to target pixel patterns.

However I think this is kind of an illusion. What `F1..n` are, are ambiguous pixel-space projections of the abstraction which isnt present in this projection. When I have the concept "this type of face" I can reason with it beyond similarity in pixel-space.

When we form representations we arent restricted to reasoning with them in only one space (eg., how faces look as pixels). We (perhaps superstitiously) impart to machine "representations" an actual depth which they lack.

They are templates derived from the spaces they live in, eg., pixel-space; and have only the properties that space affords (eg., pixel-geometry). Reasoning beyond that space, and those properties, doesnt work. People think it does. This is the illusion.

Templates derived from this data, that we provide, function like actual representations because we simplify the world for the machine -- and prepare its environment so that its pixel-space templates are good enough.

[PaulDavisThe1st]

I've been with you so far, but ..

> When I have the concept "this type of face" I can reason with it beyond similarity in pixel-space.

I think there are at least two possible things that might going on here:

1. we're "trained" on non-pixel data (to use the same framing) and so it seems obvious that we would reason with a concept like "this type of face" in a non-pixel space.

2. the experience of "reasoning with it" is an illusion, and is merely the subjective experience that we have when our brains do stuff with whatever their underlying representation is. That is, we may have no real knowledge of what space our own "face model" is built on, what it represents, what properties it considers.

[mjburgess]

So, another option, is that our concepts arent really about their targets.

For example, my "pen" is actually a bundle of largely motor techniquess (also: emotional regulation technquies etc.) which is about how I coordinate myself.

In a sense all my concepts are about me. We might say that in congition we abstract these "primal concepts" into properties which are then about their targets, sure. This is why cognition is a bit of a charlatan.

In being able to regulate my own imagination, motor skills, emotions etc. with these techniques, I can actually discover new things about the "abstract concepts" that they imply.

Since my "pen" isnt really anything like "a pen" nor even my experiences of pens... it is rather, "a way of me moving everything within me" -- I can simulate these movements and discover new things about their implied abstractions. Indeed, I can do this in the world: I can explore actual pens by moving my body differently.

This is one of the big issues with the AI paradigm as it has always existed: researchers are still talking about environments as if they had properties which were already there. Everything is formualted as-if the problem were solved. The world is just some bundle of facts (data, propositions, etc.) and representations are just subsets of these.

This misses, you know, the world. The thing that hits you. And it misses, entirely, what happens when it hits you.

[PaulDavisThe1st]

Sounds like a re-visit of a lot of the ideas in "situated action", popular in some AI circles in the 90s. That also included concepts about how we reduce cognitive load by storing information (and procedure) in our built environments.

I'm not, though, that I agree about the pen example. I mean, you're completely correct in your description of all these different elements of your "pen". But to the extent that a concept of/about something is really never anything like the thing, it's not a particularly important aspect of concepts. I think that what's important is that all of these embodied, semi-reflective elements of your "pen" concept sit in parallel with some abstract concepts about pens. Pens do have properties that are already there, but in many contexts as you note, these are less important than the ones embodied by you.

[mjburgess]

I suspect we'll find that these abstractions are no where within us. We dont have ML-like templates of stuff.

Whenever we need to reason about the properties of pens, our coordinative bodily structure generates these abstractions for cognition to operate on. They're ephemeral, and live only when cognising an issue.

To model cognition then, is to model only the symptom of intelligence; not the actual process itself.