Author here. I have never said that phrase before this blog post and certainly understand the absurdity of it. I certainly don't mean that you need something biological or whatever consciousness might or might not be.
However there's still a distinction. Unless I'm responding to an LLM, you had a childhood. You learned about the world and space and agency before you ever learned how to program. And you didn't learn it from billions of examples, you learned from a few examples, some self directed experiments, some feedback from teachers, etc...
I'm saying that's what matters. The process matters. You didn't learn to mimic a distribution, you learned to program. Of course in the perfect mathematical limit it's the same, but in practice it's not.
For a lot (most) of what we do with programming, the process actually doesn't matter. I understand you are a real ass dude who is in this shit for the love of the game. I respect that. You are a true artisan and exist in a kind of rarified space. There will always be a place for people like you and in some senses you are correct - you are not replaceable by any AI as they currently function today.
However, 99.9999% of coding is not like that. Non-coders don't care about the code at all. They just care about outcomes. People don't care if it's "slop" if it works. Similar to bug prevalence, the optimal level of slop is not zero and will be decided by the market, not by coders.
LLMs are even more useful to experts who know the limitations. However the process matters even more if you want to build robust and scalable secure systems that generate millions of dollars and can explain that accurately to high value clients.
I do not want a $10M - $100M dollar issue (lawsuits) because I admitted that I don't understand why a breach happened after using a coding agent. Responsiblity and reputation can't be vibe-coded.
So:
> However, 99.9999% of coding is not like that. Non-coders don't care about the code at all. They just care about outcomes. People don't care if it's "slop" if it works. Similar to bug prevalence, the optimal level of slop is not zero and will be decided by the market, not by coders.
There's a vast difference between code that works as a prototype vs how it works in production. I don't think you would trust anyone with no experience to fly a commercial plane with them vibe-coding a flight simulator without knowing the process of becoming a pilot.
It's not exactly what it is; they now model an incredibly complex markov process, and harnesses that control how that thinking is done.
Is this any different than how a PM gets a programmer to work on a project? They think, then they deliver. If given more time, maybe they deliver something better. Maybe they consult some text and try to apply a design pattern.
The LLM in this use case is perfect because almost everything involved is text based, and the model is able to take in all the expressive that is language.
> Is this any different than how a PM gets a programmer to work on a project?
Yes, it's very different. You seem to be suggesting that the current frontier LLMs, when tied to their tools and harnesses, have emergent properties that are similar to human consciousness. If you truly believe that, I'm not sure how to have a productive discussion here.
I think they have the capabilities to execute a well defined plan. If you truly don't believe that, then you I suspect any work you do as a programmer will not survive the coming changes.
It's not just that, but the core is just that, even with reasoning models. Harness can only get you closer to the good result, but can't save you from every pitfall.
As for PM analogy - don't forget that models don't learn and keep doing same stupid stuff they were doing a month ago.
I would suggest you examine current harness memory persistence. Any reprimand you give your model will be remembered, in the same way a puppy that has a bad social experience will become more shy.
They will not save you from every pitfall, but that isn't the point; engineers walk into pitfalls all the time. This can get you in, and out, much much quicker.
But its not useful because even humans are like that - a bunch of neurons slapped together. Overall a tired analogy that is more suited to stay in 2024 where it belongs. Right now it is clear that it is _much_ more than a statistical model semantically. It is misleading to claim it is _just_ that just like a human is _just_ a statistical model.
That's a very good way to describe what they do (better than 'AI') but ironically, it really well explains the mechanism, and how they are in fact able to 'code so well' which is contrary to the authors own premise.
Agents code extremely well.
They're not particularly good at 'architecture' and I think that's where his specific concerns about 'not being able to see the problems' arise - the issues are are almost never in the syntax, because the AI writes perfect code. The issue is that it's not doing exactly what you intended.
Instead of 'missing the target' ... it's 'hit the wrong target perfectly'.
Any senior developer working with AI daily should be able to have a baseline intuition for all of this, and would therefore reject the hyperbole of the premise 'it can't code!'.
Of course it's producing gargantuan amounts of slop - that's not because 'it can't code', that's something else entirely.
> Of course it's producing gargantuan amounts of slop - that's not because 'it can't code'
That is precisely because it can't code! Or rather, it's because it can't reason, or understand things, which in turn means it can't code. The output of LLMs is sloppy because they have no understanding of what they are doing.
That people will say 'Code Me An App' and expect some kind of magical results, will be more common than not, but it's no way evidence that the AI can't code.
Given a sufficiently detailed prompt, the AI will produce almost whatever you ask it within a certain scale.
As sure as the sky is blue.
And it will make perfectly compilable code usually on the first prompt.
Obviously, it can code.
Obviously, it can 'synthetically reason' about the code.
You can point it an arbitrary code base and it will give a better overall assessment than most humans.
Is it fallible? Obviously. Is it limited in scope? Obviously.
It's the mantra of the AI skeptics. Sounds so clever because it's technically true. Just like humans are just piles of oxygen, hydrogen, nitrogen and carbon atoms, along with maybe a dozen or so other trace elements, none of which have any intelligence or will or desire to do anything - hence humans cannot possibly be intelligent or have any free will.
However there's still a distinction. Unless I'm responding to an LLM, you had a childhood. You learned about the world and space and agency before you ever learned how to program. And you didn't learn it from billions of examples, you learned from a few examples, some self directed experiments, some feedback from teachers, etc...
I'm saying that's what matters. The process matters. You didn't learn to mimic a distribution, you learned to program. Of course in the perfect mathematical limit it's the same, but in practice it's not.