Hacker News new | ask | show | jobs
by apsec112 366 days ago
This ignores batching - token generation is much more efficient in batch - and I strongly suspect is itself written by AI, given the heavy use of bullets
2 comments

The “X—not Y” pattern is also a dead giveaway.
is it common for adjacent tokens to use the same weights in a memory cache?