[davefol]

Seems odd to claim 25x reduction in size when the algo involves looking into a database of a trillion chunks of text.

[visarga]

The "algo" here refers to the neural net itself. The text index is considered an easy problem as you can do lookups in logarithmic time.

[ehsankia]

The word "Algo" here is definitely awkward. The point is though that what matters most here is the number of parameters, as those correlate quite closely with training and inference time. Storage space is pretty trivial, but TPU cycles are less so.

[davefol]

Thanks for this. Rereading + your comment and I think I have a better understanding of why this is progress.