[mcdeltat]

It's astonishing how the two camps of LLM believers vs LLM doubters has evolved even though we as people are largely very similar, doing similar work.

Why is it that e.g. you believe LLMs are truly revolutionary, whereas e.g. I think they are not? What are the things you are doing with LLMs day to day that are life changing, which I am not doing? I'm so curious.

When I think of things that would be revolutionary for my job, I imagine: something that could input a description + a few resources, and write all the code, docs, etc for me - creating an application that is correct, maintainable, efficient, and scalable. That would solve 80% of my job. From my trials of LLMs, they are nowhere near that level, and barely pass the "correct" requirement.

Further, the cynic in me wonders what work we can possibly be doing where text generation is revolutionary. Keeping in mind that most of our jobs are ultimately largely pointless anyway, so that implies a limit on the true usefulness of any tool. Why does it matter if I can make a website in 1/10th the time if the website doesn't contribute meaningfully to society?

[bredren]

> I imagine: something that could input a description + a few resources, and write all the code, docs, etc for me

It could be that you’re falling into a complete solution fallacy. LLMs can already be great at working each of these problems. It helps to work on a small piece of these problems. It does take practice and any sufficiently complicated problem will require practice and multiple attempts.

But the more you practice with them, you start getting a feel for it and these things start to eat away at this 80% you’re describing.

It is not self driving, if anything, software engineering, automation is only accessible to those who nerd out at it, the same way using a PC used to be sending email or programming.

A lot of the attention is on being able to run increasingly capablemodels on machines with less resources. But there’s not much use to fuss over Gemini 2.5 Pro if you don’t already have a pretty good feel for deep interaction with sonnet or GPT 4o.

It is already impressive and can seriously accelerate software engineering.

[mcdeltat]

But the complete solution fallacy is what the believers are claiming will occur, isn't it? I'm 100% with you that LLMs will make subsets of problems easier. Similar to how great progress in image recognition has been made with other ML techniques. That seems like a very reasonable take. However, that wouldn't be "revolutionary", I don't think. That's not "fire all your developers because most jobs will be replaced by AI in a few years" (a legitimate sentiment shared to me from an AI-hyped colleague).

[edanm]

The thing is you're doing what a lot of critics do - lumping together different people saying different things about LLMs into one bucket - "believers" - and attributing the biggest "hype" predictions to all of them.

Yes, some people are saying the "complete solution" will occur - they might be right or might be wrong. But this whole thread with someone saying LLMs today are useful, so it's not hype. That's a whole different claim that is almost objective, or at least hard for you to disprove. It's people literally saying "I'm using this tool today in a way that is useful to me".

Of course, you also said: > Keeping in mind that most of our jobs are ultimately largely pointless anyway, so that implies a limit on the true usefulness of any tool.

Yeah, if you think most of the economy and most economic activity people do is pointless, that colors a lot about how you look at things. I don't think that's accurate and have no idea how you can even coherently hold that position.

[ryandrake]

I think the difference is between people who accept nondeterministic behavior from their computers and those who don’t. If you accept your computer being confidently wrong some unknowable percentage of the time, then LLMs are miraculous and game changing software. If you don’t, then the same LLMs are defective and unreliable toys, not suitable as serious tools.

People have different expectations out of computers, and that accounts for the wildly different views on current AI capabilities.

[mcdeltat]

Perhaps. Then how do you handle the computer being confidently wrong a large proportion of the time? From my experience it's inaccurate in proportion to the significance of the task. So by the time it's writing real code it's more wrong than right. How can you turn that into something useful? I don't think the system around us is configured to handle such an unreliable agent. I don't want things in my life to be less reliable, I want them to be more reliable.

(Also if you exist in an ecosystem where being confidently wrong 70% of the time is acceptable, that's kinda suspect and I'll return to the argument of "useless jobs")

[tinco]

Filters. If you can come up with a problem where incorrect solutions can be filtered out, and you accept that LLM outputs are closer to a correct answer than a random string then LLM's are a way to get to a correct answer faster than previously possible for a whole class of problems we previously didn't have answer generators for.

And that's just the theory, in practice the LLM's are orders of magnitude closer to generating correct answers than anything we previously had.

And then there's the meta aspect of them: they can also act as filters themselves. What is possible if you can come with filters for almost any problem a human can filter for, even if that filter has a chance of being incorrect? The possibilities are impossible to tell, but to me very exciting/worrying. LLM's really have expanded the realm of what it is possible to do with a computer. And in a much more useful domain than fintech.

[Ancapistani]

As long as it’s right more than random chance, it’s potentially useful - you just have to iterate enough times to reach your desired level of statistical certainty.

If you take the current trend of the cost of inference and assume that’s going to continue for even a few more cycles, then we already have sufficient accuracy in current models to more than satisfy the hype.

[mcdeltat]

I'm not following the statistical argument.

Firstly, something has to verify the work is correct right? Assuming you have a robust way to do this (even with humans coding it's challenging!), at some point the accuracy is so low that it's faster to create it manually than verify many times - a problem I frequently run into with LLM autocomplete and small scale features.

Second, on certain topics the LLM is biased towards the wrong answer and is further biased by previous wrong reasoning if it's building off itself. It becomes less likely that the LLM will choose the right method. Without strong guidance it will iterate itself to garbage, as we see with vibe coding shenanigans. How would you iterate on an entire application created by LLM, if any individual step it takes is likely to be wrong?

Third, I reckon it's just plain inefficient to iterate many times to get something we humans could've gotten correct in 1 or 2 tries. Many people seem to forget the environmental impact from running AI models. Personally I think we need to be doing less of everything, not producing more stuff at an increasing rate (even if the underlying technology gets incrementally more efficient).

Now maybe these things are solved by future models, in which case I will be more excited then and only then. It does seem like an open question whether this technology will keep scaling to where we hope it will be.

[mmcnl]

I guess everyone has a different interpretation of revolutionary. Some people think ChatGPT is just faster search. But 10x faster search is revolutionary in terms of productivity.