[briandw]

" With only 7M parameters, TRM obtains 45% test-accuracy on ARC-AGI- 1 and 8% on ARC-AGI-2, higher than most LLMs (e.g., Deepseek R1, o3-mini, Gemini 2.5 Pro) with less than 0.01% of the parameters"

That is very impressive.

Side note: Superficially reminds me of Hierarchical Temporal Memory from Jeff Hawkins "On Intelligence". Although this doesn't have the sparsity aspect, its hierarchical and temporal aspects are related.

https://en.wikipedia.org/wiki/Hierarchical_temporal_memory https://www.numenta.com

[java-man]

I suspect the lack of sparsity is an Achilles' heel of the current LLM approach.