[mdp2021]

> as you seem to be insinuating with

The implementation details are not clear, not the goals.

I never said that the feature has to be coded explicitly. I said it has to be there.

[skeledrew]

OK. So it's just a matter of waiting for the desired capabilities to emerge in future models.

[mdp2021]

But they will probably thought models, not just language models.

The engineering will be different.

[skeledrew]

Possibly. I personally think it's the type of data and scale that're the primary differentiators. The use of characters is a fundamental flaw because characters are synthetic entities. Instead the models should be based on raw sensory data types, such as pixels and waveforms, and iterate from there on something close to the existing architecture.