[jtbayly]

But what happens with the next generation of questions? The reason LLMs can answer how to right-align a paragraph in HTML is at least in part because it has been asked and answered publicly so many times.

Now imagine that HTMZ comes along and people just go straight to asking how to full justify text in HTMZ for their smart bucket. What happens? I doubt we’ll get good answers.

It feels like the test of whether LLMs can stay useful is actually whether we can stop them from hallucinating API endpoints. If we could feed the rules of a language or API into the LLM and have it actually reason from that to code, then my posed problem would be solved. But I don’t think that’s how they fundamentally work.

[mhuffman]

>Now imagine that HTMZ comes along and people just go straight to asking how to full justify text in HTMZ for their smart bucket. What happens? I doubt we’ll get good answers.

So, I think the answer is that since all useful data is already in a LLM somewhere all new data will be stolen/scraped and inserted in real time. So if real people are answering the question it will work as normal. The real question is what happens when people are trying to mine karma by answering questions using an LLM that is hallucinating. We have seen such with the Bug Bounty silliness going on.