[dvt]

One-shotting is a bit of a red herring, imo. Let's say I wanted to build a "Super Dario" meme platformer. I would do some research, find a platformer on Codepen or Github that "feels" good, tweak it to my liking, change the sprites, and voila.

I constantly have to tell agents to just "look it up online" and "don't hallucinate your own components" because people have already done this a million times. Ironically, being more lazy could make these models more useful.

[bogwog]

So you're saying that it's not fair to rate a coding model on its ability to code, and instead the best way to use it is to tell it to find existing human-written code online rather than generate actual code on its own?

[dvt]

If you're testing how good LLMs are at compressing information, then I think that's a fair test. Personally, I don't really think that's where their strength comes from (especially considering how much more useful local models that are orders of magnitude smaller than Claude/OpenAI-tier models have gotten). In other words, we already have a "super-intelligence"—it's called the internet, so just use the darn thing.

[darkwater]

Isn't part of the human coding that Super Dario tells us LLM are going to replace also made of what you are saying GP is saying?

[sefrost]

Yes for any slightly non-standard web UI components I either tell the LLM to search the web and find examples or I provide them myself.

Usually the ones I provide myself are better than the ones they find themselves.

I'm not super clear how they search the web or what they do and don't have access to. It seems to be quite fast.

[port11]

Strange, I use M3 and Sonnet a lot, they never hallucinated. Gemini used to make up some function names, but it was in long context windows. My AGENTS.md does specific they should use context7 when unsure about a library, but still…

[jimkleiber]

TBH, I wonder how much one-shotting posts are just lying about one-shotting it.

[uncivilized]

Many for sure.