Hacker News new | ask | show | jobs
by antirez 491 days ago
The relation among the internal model representations inside its latent space and the embedding of the CoT compressed with a text embedding model is, more or less, minimal. Then we take this and map it to a 2D space, which captures more or less nothing of the original dimentionality and meaning. That's basically plotting random points.
1 comments

Potentially it's useful to understand a model "on its own terms" via its observable outputs.

>The relation among the internal model representations inside its latent space and the embedding of the CoT compressed with a text embedding model is, more or less, minimal.

This may or may not be correct but one way to find out is by taking a look!