Hacker News new | ask | show | jobs
by DoctorOetker 35 days ago
I believe it is possible to "upload" the knowledge from future reformulations of LLM's.

Right now most of the weights are in the FFN neural nets, but I believe we can reformulate LLM's or distill them into a format where each token/character represents either a matrix or a geometric algebra multivector. If each weight becomes token/character associated, it means we could write mindless reaction speed games: have a 2D projection of token/word/concept multivector/matrix coordinates. Let the 2D projection smoothly evolve a grand tour through the space of projections, possibly only including new dimensions if user performance on the explored dimensions suffices. 95% of the tokens/words could be rendered at their correct positions, while say 5% is at their incorrect positions. So the user sees a subset of the character/word/tokens rendered, and since they are individually placed at a correct position 95% of the time, it allows a user to learn peripherally, the user just needs to click those tokens intentionally disturbed from their true positions. Consider the hypothetical scenario that eventually all coordinates are explored and hypothetically the user scores highly (perhaps 100% or 90%) then we could say that at least the high dimensional coordinates are properly uploaded. Now you could randomly render bigrams (the new matrix coordinates being the matrix product / the new multivector coordinates being the geometric product). Next we could hypothesize that with sufficient training the user can predict the correct position of bigrams (meaning the user brain is performing the same or similar computation). From this the likelihood could be computed, and so a user would start predicting the same next tokens the model would. It wouldn't be blind belief more like a user associating the claims with what the reaction speed game insinuates. Imagine having trained on random cyrillic characters and suddenly you understand the Greek ingredients on some food packaging. There is a lot of hypothesis here, but none of the steps seem impossible to me.

It would mean both kids and adults could sync up with what LLM's claim (without blindly believing in it, but when it helps predict a lot of things around you people may lean and trust more on it, so that danger is not entirely moot), and having an LLM acquaintance in your brain to consult.