[alganet]

Does that really matter?

He is trying to lax the general public perception around AIs shortcomings. He's giving AI a break, at the expense of regular developers.

This is wrong on two fronts:

First, because many people foresaw the AI shortcomings and warned about them. This "we can't fix a bug like in regular software" theatre hides the fact that we can design better benchmarks, or accountability frameworks. Again, lots of people foresaw this, and they were ignored.

Second, because it puts the strain on non-AI developers. It blamishes all the industry, putting together AI with non-AI in the same bucket, as if AI companies stumbled on this new thing and were not prepared for its problems, when the reality is that many people were anxious about the AI companies practices not being up to standard.

I think it's a disgraceful take, that only serves to sweep things under a carpet.

[SalientBlue]

I don't think he's doing that at all. The article is pointing out to non-technical people how AI is different than traditional software. I'm not sure how you think it's giving AI a break, as it's pointing out that it is essentially impossible to reason about. And it's not at the expense of regular developers because it's showing how regular software development is different than this. It makes two buckets, and puts AI in one and non-AI in the other.

[alganet]

He is. Maybe he's just running with the pack, but that doesn't matter either.

The fact is, we kind of know how to prevent problems in AI systems:

- Good benchmarks. People said several times that LLMs display erratic behavior that could be prevented. Instead of adjusting the benchmarks (which would slow down development), they ignored the issues.

- Accountability frameworks. Who is responsible when an AI fails? How the company responsible for the model is going to make up for it? That was a demand from the very beginning. There are no such accountability systems in place. It's a clown fiesta.

- Slowing down. If you have a buggy product, you don't scale it. First, you try to understand the problem. This was the opposite of what happened, and at the time, they lied that scaling would solve the issues (when in fact many people knew for a fact that scaling wouldn't solve shit).

Yes, it's kind of different. But it's a different we already know. Stop pushing this idea that this stuff is completely new.

[SalientBlue]

>But it's a different we already know

'we' is the operative word here. 'We', meaning technical people who have followed this stuff for years. The target audience of this article are not part of this 'we' and this stuff IS completely new _for them_. The target audience are people who, when confronted with a problem with an LLM, think it is perfectly reasonable to just tell someone to 'look at the code' and 'fix the bug'. You are not the target audience and you are arguing something entirely different.

[alganet]

Let's pretend I'm the audience, and imagine that in the past I said those things ("fix the bug" and "look at the code").

What should I say now? "AI works in mysterious ways"? Doesn't sound very useful.

Also, should I start parroting innacurate outdated generalizations about regular software?

The post doesn't teach anything useful for a beginner audience. It's bamboozling them. I am amazed that you used the audience perspective as a defense of some kind. It only made it worse.

Please, please, take a moment to digest my critique properly. Think about what you just said and what that implies. Re-read the thread if needed.

[beyarkay]

> He is trying to lax the general public perception around AIs shortcomings

This is not at all what I'm trying to do. This same essay is cross-posted on LessWrong[1] because I think ASI is the most dangerous problem of our time.

> This "we can't fix a bug like in regular software" theatre hides the fact that we can design better benchmarks, or accountability frameworks

I'm not sure how I can say "your intuitions are wrong and you should be careful" and have that be misintepreted as "ignore the problems around AI"

[1]: https://www.lesswrong.com/posts/ZFsMtjsa6GjeE22zX/why-your-b...

[alganet]

The word intuition here have power. Surely you must have noticed that vibe coding and intuition have a strong connection.

There are people (definitely not me) who buy 100% of anything AI they read. To that beginner enthusiastic audience, your text looks like "regular software is old and unintuitive, AI is better because you can just vibe", and the text reinforces that sentiment.