So you're saying that it's not fair to rate a coding model on its ability to code, and instead the best way to use it is to tell it to find existing human-written code online rather than generate actual code on its own?
If you're testing how good LLMs are at compressing information, then I think that's a fair test. Personally, I don't really think that's where their strength comes from (especially considering how much more useful local models that are orders of magnitude smaller than Claude/OpenAI-tier models have gotten). In other words, we already have a "super-intelligence"—it's called the internet, so just use the darn thing.