Hacker News new | ask | show | jobs
by mdp2021 304 days ago
But they will probably thought models, not just language models.

The engineering will be different.

1 comments

Possibly. I personally think it's the type of data and scale that're the primary differentiators. The use of characters is a fundamental flaw because characters are synthetic entities. Instead the models should be based on raw sensory data types, such as pixels and waveforms, and iterate from there on something close to the existing architecture.