Hacker News new | ask | show | jobs
by internet_points 80 days ago
Also tokens don't represent meaning in themselves, but are assigned points in a multidimensional space, they can only represent meaning in the network as a whole when combined with other tokens in context and order.

And the abstract concepts of Abstract Wikipedia are human-defined, top-down ways of carving the world into distinct categories which make some kind of logical sense, whereas LLM's work bottom-up and create overlapping, non-hierarchical, probabilistic networks of connections with nearly no imposed structure except the principle that you shall know a token by the company it keeps.

But you can type them both out with keys on a keyboard so in that sense I guess they're not that different.