Hacker News new | ask | show | jobs
2D Tokenization for Large Language Models (david-gilbertson.medium.com)
1 points by bubble_boi 1149 days ago
1 comments

This article is about how we process text before passing it to large language models, the problems with this approach, and an alternative solution.