[PBnFlash]

The way these systems work feel massively inefficient.

We don't re-evaluate our astrophysics models when reading a cooking book.

[DavidSJ]

Neither does GPT-4 or other sparse mixtures of experts, such as e.g. switch transformers [1].

[1] https://arxiv.org/abs/2101.03961

[rini17]

Speak for yourself :)