[lumost]

although, the correct answer is also likely on the web. With a suitable search query you would see the correct paper/textbook/wiki page with the right answer. A text highlighting model could also likely extract this answer from the text. The training probably achieves a good degree of memorization for these known results.

This begs the question, would we be impressed with a similar compression algorithm for storing past web documents?

[amelius]

The main achievement is not the compression, but the search functionality (search==solve).

[kortilla]

Well the trivial test to make sure it’s not memorized would be to change constants in the input that alter the correct answer but don’t make the problem any more difficult if it is actually doing the calculation.