[dannersy]

Reading this thread makes me feel like I'm taking crazy pills. The folks on this train in my team do not produce anything significant that we can rely on or use. A lot of hollow prototypes that join the prototype graveyard and code that needs extra scrutiny on critical areas ultimately leading to taking longer.

It's a shame, they were smart and productive engineers. Now? I guess everyone is just all-in on the slot machine.

[frumiousirc]

This split in what different people or groups get out of LLMs is pervasive and really interesting. In the beginning I was dismissive of those with bad experience with a "you are holding the tool wrong" smugness. But as I read more and more experience, I see all combos and I now know my initial knee jerk conclusion was clearly wrong. There are newbie programmers getting good or bad results as well as experienced developers getting either flip of the coin. I don't know what to conclude. I really want to know what are the lines that explain these very different outcomes. Is it the types of problems being solved? The harnesses? The programming languages? FWIW, my experience has been that among my cohorts of mid to deeply experienced developers working in the domain of experimental physics, all have leveled up various degrees after adopting Sonnet and Opus level LLMs using claude code CLI in Python, C++ and web tech, small scale scripts up to multi-package novel system develop and green field as well as incremental development and code maintenance.

[dannersy]

I have seen plenty of greenfield projects go okay at first but never go the distance. These were mostly product software cases, where they were able to get something very professional looking very fast but AI ultimately always miss the mark because they are taking the median of what exists and not the specific needs of the customer they're developing for. So they get a ton of features and few that were necessary, then developing it further and correcting it to the needs of the customer just makes a mess and regressions are frequent. This is my experience as well when it comes to being a consumer of software products, everything feels shittier and less reliable, perhaps that's my emotion and bias coming out.

The last 20% of the software development cycle is always the hardest. Releasing, maintenance, usability, support. You know, having a real product. I don't see AI helping here at all, more the first 80%, which sadly is also the fun part.

When developing things that are novel, with designs specific to our use cases needing high throughput, the results are pretty dismal. AI can kind of get you there, but I've seen no advancement on this front with new models. At the end of each attempt we've always realized we should have done things by hand. Having people with intense knowledge of the system frequently comes from building it and troubleshooting it, I don't think serious engineering orgs have escaped this inevitability.

On cases where we have legacy software, AI has helped with understanding shit code and design, but woefully bad at contributing to legacy software. Here be dragons for sure. It is super strange to me that these tools can seemingly easily diagnose but completely blunder the fix.

I could easily see there being gains, as you say, in fields where data wrangling becomes tedious (though the inherent error rate in AI outputs scares me if you're trying to get deterministic outputs from experiments... I digress).

The part I think this forum tends to forget, and the tech industry at large fails to even care about, is that we're still humans. There are many studies basically pointing out that the way the AI outputs information is bad for us. Instant gratification from anthropomorphised machines with a habit for sycophancy doesn't sound like a recipe for a healthy relationship with what everyone wants to claim is just a tool. AI providers know this is effective, as well as knowing that there is a gambling effect here. They care about making money, not a good product and they happily prey on our human weaknesses. That is what social media is now. They aren't good products anymore, they just promote addiction via engagement.

Sorry for the long response and cynicism, but that is just my anecdotal experience and perspectives. I can give sources to some of the objective claims if you want.