[oxfordmale]

There remains a significant challenge with LLM-generated code. It can give the illusion of progress, but produce code that has many bugs, even if you craft your LLM prompt to test for such edge cases. I have had many instances where the LLM confidentially states that those edge cases and unit tests are passing, while they are failing.

Three years ago, would you have hired me as a developer if I had told you I was going to copy and paste code from Stack Overflow and a variety of developer blogs, and glue it together in a spaghetti-style manner? And that I would comment out failing unit tests, as Stack Overflow can't be wrong?

LLMs will change Software Engineering, but not in the way that we are envisaging it right now, and not in the way companies like OpenAI want us to believe.

[vidarh]

Proper coding agents can easily be set up with hooks or other means of forcing linting and tests to be run and prevent the LLMs from bypassing them already. Adding extra checks in the work flow works very well to improve quality. Use the tools properly, and while you still need to take some care, these issues are rapidly diminishing separately from improvements to the models themselves.

[scubbo]

> Use the tools properly

> (from upthread) I was being sold a "self driving car" equivalent where you didn't even need a steering wheel for this thing, but I've slowly learned that I need to treat it like automatic cruise control with a little bit of lane switching.

This is, I think, the core of a lot of people's frustrations with the narrative around AI tooling. It gets hyped up as this magnificent wondrous miraculous _intelligence_ that works right-out-of-the-box; then when people use it and (correctly!) identify that that's not the case, they get told that it's their own fault for holding it wrong. So which is it - a miracle that "just works", or a tool that people need to learn to use correctly? You (impersonal "you", here, not you-`vidarh`) don't get to claim the former and then retreat to the latter. If this was just presented as a good useful tool to have in your toolbelt, without all the hype and marketing, I think a lot of folks (who've already been jaded by the scamminess of Web3 and NFTs and Crypto in recent memory) would be a lot less hostile.

[TeMPOraL]

How about:

1) Unbounded claims of miraculous intelligence don't come from people actually using it;

2) The LLMs really are a "miraculous intelligence that works right out-of-the-box" for simple cases of a very large class of problems that previously was not trivial (or possible) to solve with computers.

3) Once you move past simple cases, they require increasing amount of expertise and hand-holding to get good results from. Most of the "holding it wrong" responses happen around the limits of what current LLMs can reliably do.

4) But still, that they can do any of that at all is not far from a miraculous wonder in itself - and they keep getting better.

[scubbo]

With the exception of 1) being "No True Scotsman"-ish, this is all very fair - and if the technology was presented with this kind of grounded and realistic evaluation, there'd be a lot less hostility (IMO)!

[vidarh]

The problem with this argument is that it is usually not the same people making the different arguments.

[scubbo]

That would be true if I was making a argument of criticism of a certain person or class-of-people, but I'm not. I'm describing my observations about the frustrations that AI-skeptics feel when they are bombarded with contradictory messages from (what they perceive as) "the pro-AI crowd". The fact that there are internal divisions within that group (between those making absurd claims, and those pointing out how correct tool use is important) does mean that the tool-advisers are being consistent and non-hypocritical, but _it doesn't lessen the frustration by the people hearing it_.

That is - I'm not saying that "tool advisers" are behaving badly, I'm observing why their (good!) advice is met with frustration (due to circumstances outside their control).

EDIT: OK, on reading my previous comment, there is some assumption that the comments are being made by the same people (or group-of-people) - so your response makes sense as a defence against that. I think the observations of sources of frustration are still accurate, but I do agree that "tool advisers" shouldn't be accused of inconsistency or hypocrisy when they're not the ones make outlandish claims.