Hacker News new | ask | show | jobs
by akarshkumar0101 391 days ago
We mention this issue exactly in the fourth paragraph in Section 4 and in Appendix F!
1 comments

That is addressing the incomprehensibility of PCA and applying a transformation to the entire latent space. I've never found PCA to be meaningful for deep learning. As far as I can tell, polysemous issue with neurons cannot be addressed with a single linear transformation. There is no sparse analysis (via linear probes or SAEs) and hence the unaddressed issue.