[imiric]

> What are we doing?

Building tools that enable people with no experience to create and ship software without following any good software engineering practices.

This is in no way comparable to any previous period in the industry.

Education and support are more accessible than ever. Even the tools used to create such software can be educational. But you can't force people to learn when you give them the tools to create something without having to learn. You also can't blame them for using these tools as they're marketed. This situation is entirely in the hands of AI companies. And it's only going to get worse.

The only thing experienced software developers outside of the AI industry can do is observe from the sidelines, shake our heads, and get some laughs out of this shit show. And now we're the bad guys? Give me a break.

[larve]

A computer always was a tool to enable people without technical knowledge to build software. That was true for me as 9 year old in the 80ies.

LLMs are incredible engineering tools and brushing them aside as nonsense is imo doing a disservice to everybody, and especially ourselves if we take our craft seriously. You can literally replace llm with php and post the same take on usenet in 1999, or whenever you started writing software.

I am tired of engineers just throwing their hands up and being defeatist while fully endorsing whatever narratives the ai industry is throwing out there, when what we are talking about is a big pile of floats that is able to generate something that makes it into the App Store. It is unprecedented in its abilities, but it’s also nothing new conceptually. It makes computer things easier.

[imiric]

> A computer always was a tool to enable people without technical knowledge to build software.

That's just not true.

Every past technology that claimed to enable non-technical people to build software has either failed, or was ultimately adopted by technical people. From COBOL, to BASIC, to SQL, to Low-Code, to No-Code, and others. LLMs are the latest attempt at this, and so far, they've had much more success and mainstream adoption than anything that came before.

The difference with LLMs is that it's the first time software can be built and deployed via natural language by truly anyone. This is, after all, their most advertised feature. The skills required to vibe code are reading and writing English, and basic knowledge to use a modern computer. This is a much lower skill requirement than for using any programming language, no matter how user friendly it is. Sure, there is a lot of poor quality software today already, but that will pale in comparison to the software that will be written by vibe coding. Most of the poor quality software before LLMs was limited in scope and reach. It would never have been deployed, and it would remain abandoned in some GitHub repo. Now it's getting deployed as quickly as it can be generated. "Just fucking ship it."

> LLMs are incredible engineering tools and brushing them aside as nonsense is imo doing a disservice to everybody

I'm not brushing them aside as nonsense. I use these tools as well, and have found them helpful at certain tasks. But there is a vast difference from how domain experts use these tools, and how the general public uses them. Especially people who are only now getting into software development, and whose main interest is to quickly cash out. If you think these people care about learning best software development practices, you'd be sorely mistaken. "Just fucking ship it."

[larve]

I don't think that COBOL, BASIC, SQL have failed. They allowed many non-technical people to get started building things with computers. The skills to vibe-code (or more generally building applications with LLMs) are not reading and writing english, they are the skill of using LLMs to build applications.

In the context of people not learning "real programming", you can equate LLMs to say, wordpress plugins or making a squarespace site. Deployment of software has never been gated by how much effort it took to write it, there's millions of wordpress sites out there that get deployed way faster than an LLM can generate code.

If we care about the security of it all, then let's build the platforms to have LLMs build secure applications. If we care about the craft of programming, whatever that means in this day and age, then we need to catch people building where they are. I'm not going to tell people to not use computers because they want to cash out, they will just use whatever tool they find anyway. Might as well cash out on them cashing out while also giving them better platforms to build upon.

As far as the OP goes, these kind of security issues due to hardcoded credentials are basically the hallmark of someone shipping a (mobile|web) app for the first time, LLMs or not. The only reason the LLM actually used that is because it was possible for the user to provide it tokens, instead of replit/lovable/expo/whatever providing a proper way to provision these things.

Every cash~out fast bro out there these days uses stripe and doesn't roll their own payment processing anymore. They certainly used to do so because they just clicked a random wordpress plugin. That's what I think a more productive way to tackle the issue is.

[imiric]

> I don't think that COBOL, BASIC, SQL have failed. They allowed many non-technical people to get started building things with computers.

Those didn't fail, but they're certainly not used by non-technical people. That was my point: that all technologies that previously promised to make software development accessible for non-technical people didn't deliver on that promise, and that they're used by software engineers today. I would chalk up the Low-Code and No-Code tools as general failures, since neither business people nor engineers want to use them.

> In the context of people not learning "real programming", you can equate LLMs to say, wordpress plugins or making a squarespace site.

I don't think that's an accurate comparison, as website builders only cover a small fraction of what's possible with "real programming". Web authoring and publishing tools have existed since the dawn of the web, and the modern ones simply turned it into a service model.

LLMs OTOH allow creating any type of software (in theory). They're much broader in scope, and lower the skill requirements to create general-purpose software much more than any previous technology. The software in TFA was an iOS app. This is why they're a big deal, and why we're seeing scam artists and grifters pump out these low-effort applications in record time and volume. They were already enabled by WordPress and Squarespace, and there are certainly a lot of scam and spam sites on the web thanks to website builders, but their scope, reach and productivity got multiplied by LLMs.

> If we care about the security of it all, then let's build the platforms to have LLMs build secure applications.

That's easier said than done, if it's possible at all. Security, privacy, and bug-free software is not something that can be automated, at least with current technology. It requires great care and attention to detail from expert humans, which grifters have zero interest in, and non-expert non-grifters don't have the experience or patience to do. Vibe coding, after all, is the idea that you keep pasting errors to the LLM and prompting it until the software on the surface works as you expect it to. Code is just the translation layer for the LLM to write and interpret; vibe coders don't want to know about it.

Could we encode some general security and privacy hints in the LLM system prompt so that it can check for specific issues? Sure. It will never be exhaustive, though, so it would just give a false sense of security.

> As far as the OP goes, these kind of security issues due to hardcoded credentials are basically the hallmark of someone shipping a (mobile|web) app for the first time, LLMs or not.

Agreed. What I think you're not taking into account is the fact that there is a large swath of the population who just doesn't care about this. The only thing they care about is having an easy way to pump out a service that attracts victims who they can quickly exploit in some way. Once that service is no longer profitable, they'll replace it with another. What LLMs have given these people is an endless revenue stream with minimal effort.

This is not the same group of people who cares about software, the product they're building, and their users. Those are a small minority of the new generation of software developers who will seek out best practices and figure out how to use these tools for good. Unfortunately, I don't think they will ever become experts at anything other than interacting with an LLM, but that's a separate matter.

So the key point is: building high quality software starts with caring. Good practices that ensure high quality are discovered by intentionally seeking out established knowledge, or by trial and error. But the types of issues we're seeing here are not because the developer is inexperienced and made a mistake—it's because they don't care. Which should be criticized and mocked in public, and I would argue regulated and fined, depending on the severity. I even think that a software development license is even more important today than ever before.