[w_t_payne]

I found the most interesting part was the evocative comment about the 'vast and parched' nature of the latent space.

I wonder if the OP's intuition regarding the sparseness of the latent space, and the relatively small area occupied by the 'useful' manifold? embedded within it provide us any clues as to what symbol grounding might look like for some neuro-symbolic infrastructure that sits atop that latent space.

I.e. how should we be trying to represent concepts like 'male' and 'female' within that space?

Is it important to have these concepts represented as a low dimensional manifold?

Is it important that this manifold be easily described by some simple geometric form like a convex polytope?

Is it important that nuances and variations on the concept be separable within the bounds of the concept-specific manifold?

What other properties might be important?