[visarga]

Apparently GPT-3 trained on code gained the ability for reasoning, but plain language, or even multi-task fine tuning, were not enough. Maybe the quality of the data makes a difference in the robustness of the model.

This interesting discovery, when applied to brains, means the same brain trained with different data would be able to display emerging abilities. Maybe these abilities are more in the data than the architecture.

If we want better AI we need to come up with better data.

[seydor]

Maybe, or most of human writing is rambling without much effective structure, unlike code