Hacker News new | ask | show | jobs
by zaptrem 220 days ago
They don't have prefix caching? Claude and Codex have this.
1 comments

At those speeds, it's probably impossible. It would require enormous amounts of memory (which the chip simply doesn't have, there's no room for it) or rather a lot of bandwidth off-chip to storage, and again they wouldn't want to waste surface area on the wiring. Bit of a drawback of increasing density.