[azakai]

> All of the AI projects we have observed as a team are failing. Every single one – we have seen 0% success in a year and a half,

What is an "AI project"? The post doesn't define it.

Is it writing some software from scratch? Using an LLM chatbot by non-coders, either internally or externally? Or something else entirely?

Some examples would really help.

[jdlshore]

Their company does data projects. That plus context makes me think they’re talking about internal work process automation type of work, although it also seems like they’re talking about conversational interfaces (chatbots).

I completely buy the “emperor’s new clothes” argument for work process automation. I’m surprised they don’t address AI-assisted engineering, which seems to be going positively for a lot of folks (although I have doubts about its sustainability). I disagree about the success of chatbots, if the problem is narrowly-defined and chosen properly. My previous company built a conversational interface to a vector database and saw good results. (Although, arguably, the vector database was the real magic, and a traditional UI would have been faster and more accurate.)

In general, I think OP is more right than wrong, though, particularly about the AI mania and unrealistic expectations sweeping the C-suite.

[listic]

Do yo have links handy for AI-assisted engineering going positively? The case I have on my mind of it going negatively is this recent Ford case [1] It's not that I believe it couldn't go positively, of course.

[1] Ford rehires human engineers after AI fails to match quality checks

https://www.bbc.com/news/articles/cgrkd41n2v9o

[simonw]

That Ford story was really misleading. It wasn't about modern LLMs, and the way it was reported implied that Ford had fired and then hired people but if you read closer that wasn't necessarily the case at all - it sounded more like they were re-hiring people who had retired because they needed expertise that had left the company.

You need to get through the Bloomberg paywall: https://www.bloomberg.com/news/articles/2026-06-25/ford-has-...

> Over the last three years, Ford says it has hired 350 veteran engineers, many of them former employees and others from suppliers, to help address seemingly intractable quality woes that have cost the automaker billions. [...]

> “We had been relying more and more on automated quality systems” and not getting the desired results, Galhotra said. “We brought back technical specialists” and “they hunt for failure points before a part ever reaches the plant floor.”

(I made these points on the HN thread about it 3 weeks ago and got voted down and I'm still salty about it https://news.ycombinator.com/item?id=48674446#48675045 )

[daveguy]

> That Ford story was really misleading. It wasn't about modern LLMs, and the way it was reported implied that Ford had fired and then hired people but if you read closer that wasn't necessarily the case at all - it sounded more like they were re-hiring people who had retired because they needed expertise that had left the company.

Oh come on, Simon. Are your glasses so rose coloured they are opaque? They fired, and then re-hired because the AI couldn't do the job. They had to re-hire the quality folks. So, no, it wasn't LLMs, it was quality control -- which is a much more well established domain for automated systems. LLMs are a complete shitshow compared to industrial process automation.

Side question -- do you worry about being so pro-LLM when the promises of LLMs are so clearly falling short?

[simonw]

Show me evidence that they had previously fired any of the people who were hired.

Your comment is exactly the misinterpretation I'm pushing back against here.

The time period covered by this story starts three years ago! What kind of LLMs do you think they were using back then?

> Side question -- do you worry about being so pro-LLM when the promises of LLMs are so clearly falling short

I think my record is looking pretty good here. I was early to the "LLMs are useful for writing code" thing, especially with the code interpreter pattern (both write and then execute code in a loop) which I now realize was our first hint at coding agents.

A couple of years ago I was one of the few people talking about what a natural fit LLMs were for the commandline - https://simonwillison.net/2024/Jun/17/cli-language-models/ With hindsight maybe I should have doubled-down on that!

I've also written plenty about the weaknesses of these models - in terms of security in particular - which has aged well.

[daveguy]

3 years ago is about when the BS "AI can do anything!" hype started. So, it is not a misinterpretation, but an accurate assessment of the hype cycle. Exactly the kind of things managers are bumbling into because "AGI any day now!" for the last 3-4 years. GPT 3.5 was released May 2022. I'm curious how many AI project in industry have crashed and burned during this hype-Summer before the Winter. Speaking of which, whatever happened to that "software dark factory" you were gushing over a while back?

[aragilar]

I think it's not clear from the article why they left (e.g. could be anything from retirement to going to work at another firm/contracting to being fired to switching careers), and likely it's going to be a mix, plus not all were previous Ford employees. Similarly the "AI" isn't clearly defined (but like you I would be surprised if it were LLMs). I suspect though why the article exists (and a possible source of your downvotes) is signalling against "AI", which if Ford wants more expert employees (given their issues), is something Ford wants to present.

[jdlshore]

As @simonw said, the Ford example isn’t a good one.

As for AI-assisted engineering going well, I think the jury is still out. Here on HN and with the engineers I know, you see people claiming multiples of productivity on coding tasks. But you also see people complaining about drowning in slop PRs.

I think there’s a lot of confounding factors to these reports. The type of work matters a lot: bug fixing good, prototyping good, big legacy codebases not so much, but maybe good for increased understanding. The type of automation matters: aggressive autocomplete good, vibe coding bad, dark factory (vibe coding with fancy harnesses and auto-“correcting” eval loops) questionable.

And then finally, the perennial mistake our industry makes, which is to value speed of creation over maintenance costs. Personally, I think this is where AI-assisted engineering is going to fall down really hard, but the jury’s still out on that one.

Anyway, there’s a really big spread in experiences with AI, that I think chalk up more to all this context rather than religion and belief. OP didn’t address it at all, which I think is a big gap in their essay, but I do think think they describe the executive-level mania pretty well.

[ethin]

> As for AI-assisted engineering going well, I think the jury is still out.

Anecdotally, AI-assisted engineering has helped me flesh out ideas or to learn extremely complicated APIs faster than trying to understand the docs (which usually are labyrinthine). MS COM ones, for example. I can go read the docs but it's easier to get a quick idea of what I need to do if I ask Claude to provide me an example of doing something specific with it, because MS's code samples (particularly their full ones in, say, the windows Desktop SDK repo) have always been annoying for me to wade through because I have to filter out a bunch of noise. I can't (and won't) try to guestimate "productivity" improvements though, but as an assistant AI has (somewhat) helped. I still do all the engineering work though. Along with it giving me tips on using more modern language features for languages like C++.

[badsectoracula]

The Microsoft API docs are a special case where i'd say you pretty much need LLMs nowadays because after a bunch of document format conversions over the years they degraded massively.

If you can find some MSDN CDs/DVDs from the early 2000s, the content is much better (and you can clearly see that the current docs are often missing descriptions and names for method arguments or even entire paragraphs).

[ethin]

What places would you suggest where I can get digital copies of that? I don't have a physical CD/DVD reader so... :)

[inigyou]

LLMs are good at natural language search. They're bad at everything else.

[prodigycorp]

The Ford case is not about AI coding. It's about computer vision processes that went wrong. This was less about AI and more about Ford being dumb.

[skydhash]

> I disagree about the success of chatbots, if the problem is narrowly-defined and chosen properly.

If you can narrow the problem down, then you could design a much better interface for it than a text box and free form text (unless that's the better solution).

As for as AI assisted engineering goes, the thing is that after some time with a project, you already have much of the workflows and routines nailed down as scripts and other various combinations of tooling. And unless it's spaghetti code, you will have various snippets you can copy from for new code. The one thing I've observed about AI projects is that there's often little technical design coherence about them. It's always a kitchen sink of technologies and practices.

[jdlshore]

> If you can narrow the problem down, then you could design a much better interface for it than a text box and free form text (unless that's the better solution).

Yes, I agree, in that the chatbot we built probably would have worked just as well with a traditional UI, and would have been done a lot faster. But it would have been a lot less sexy (actually important for the bottom line!) and there are future directions that could take advantage of the conversational interface that’s potentially better than a traditional UI.

On the down side, good chatbots are really frikkin difficult to write. These things (LLMs) are not reliable at scale. The basic functionality came together in weeks. Getting it to behave consistently and obey guardrails took months, and even then we had to accept a low level of failed conversations.

[skydhash]

> But it would have been a lot less sexy (actually important for the bottom line!

That’s what the author have been saying. They do for nice demos which sell the illusion of having Jarvis in text format, but the usefulness is not really proven. And that may be important business wise. But as far as end users is concerned, there’s not a lot of productivity boost.

[Aurornis]

They run a consulting company called "Hermit Tech". Their websites has an olde style font.

They boast about "ancient techniques" from books written prior to the year 2000

> For non-executive management who might be struggling to deliver things that feel beyond their control, we have ancient techniques (see: books written between 1986 and 1999) to turn your team into the envy of the organisation, and we can drop in directly to get your team the resources it needs to save a struggling project.

So yeah, of course these people hate AI and everything about it.

No serious company is reaching out to these people for help with their AI project.

[parpfish]

i feel like the useful applications of AI get silently integrated into workflows.

the ill-conceived moonshots by and for a non-technical audience get labelled as "AI projects/initiatives" and they fail.

[jaggederest]

That has ever been the case. As soon as it works reliably, it's not "AI" any more. Take spellcheckers or collaborative filtering as examples, but there are lots more. Hofstadter in G.E.B. said it well:

> There is a related “Theorem” about progress in AI: once some mental function is programmed, people soon cease to consider it as an essential ingredient of “real thinking”. The ineluctable core of intelligence is always in that next thing which hasn’t yet been programmed. This “Theorem” was first proposed to me by Larry Tesler, so I call it Tesler’s Theorem: “AI is whatever hasn’t been done yet.”

[jerf]

In this moment, the opposite is happening. Everything is getting called "AI", whether it uses LLMs to prompt LLMs about how to prompt LLMs, uses "conventional" machine learning, or just looks mysterious enough that they can expect the market to not ask questions.

I am reminded of "game AI", which for the most part has historically been just giant decision trees, encoded one way or another, because if you hook up any sort of real AI to a game entity or collection of game entities that does any sort of learning or training, even simple 1980s-era reinforcement learning, it turns out the game entities will roflstomp the human players, and the human players aren't interested in paying for that experience. We've been calling those collections of if statements and for loops "AI" for a long time, though, because who wants to hear about how deliberately stupid their opponents are?

[ambicapter]

Sure, or maybe the actual applications of “AI” are small and unobtrusive, like the dictation of doctor’s notes example, and it’s not actually the massive revolution it’s claimed to be.

[jaggederest]

I struggle to reconcile this attitude with, among other things, the massive number of exploits being discovered and the pretty straightforward utility for coding. Have you used codex recently?

[variadix]

Perhaps, but I think what people mean by intelligence is something that learns and adapts. If LLMs couldn’t do in context learning I don’t think people would think of them as AI, more as a kind of queryable database via natural language. There are other algorithms that learn and adapt, but in much more narrow circumstances that most people won’t obviously interact with.

[jaggederest]

Right, and my point is that now that is the bar. If you'd told me in 2008 that markov chains would be smart enough to one-shot even a trivial video game from scratch I would certainly have called that AI even if you had to do some major rigamarole to get it to work.

[inigyou]

Because AI meant "make computers smart like people".

We taught the computer to spell check, but the computer still didn't feel smart like a person, just smart like a machine so that obviously wasn't AI. We taught it to do algebra, same thing. With LLMs though, now it really does feel like an artificial human, so this time it really is AI.

[jaggederest]

Right, but I remember when spellcheck first started to be a thing, and people were like "Wow proofreaders are out of a job, ai is so cool", but then it became normal.