[threethirtytwo]

You're all in denial when you criticize LLMs. It's not necessarily that the criticism isn't true. It's more how self assured the criticism is. That's the biggest problem because AI is a moving target. It is getting better, and it is getting better fast. A lot of the criticism can become outdated in a year or six months. The change is happening in front of your very eyes and yet you can always reliably come on HN and find some sort of self assured criticism to say AI can't design, AI code must always be reviewed. Blah blah blah.

The big thing people used to call AI was that it was a stochastic parrot and all it did was summarize things. Clearly. None of this is/was true anymore. And very likely all the current criticism will be eliminated soon and we have to find new excuses about AI that makes us feel we are superior.

The status quo is about to change. Every 6 months. And you will always think of yourself as superior to LLMs. Your current criticisms will evolve as most of them will be rendered not true pretty soon.

[adamas]

> AI code must always be reviewed

Yeah, of course AI code must always be reviewed. All code must be reviewed.

[threethirtytwo]

Nope. Not true anymore. Right now the bottleneck is the review because code comes out so fast there is a measurable and adjustable trade off that can be made.

If you review all code, your output will be slow. If you review less code, your output will be faster at the cost of more bugs in production. Bug rate will never go down to zero whether you use AI or not.

That is the trade off, if you review everything then output is really slow. If you start only reviewing certain types of code like model changes, database changes. Or only backend code and not frontend code, you hit a sweet spot of speed and reliability.

As LLMs improve the need for reviews becomes less and less. Companies who don't adjust are just slowing themselves down. That is the trend.

[adamas]

With that logic, you would let an intern force push his way to prod, that's not a smart move.

[threethirtytwo]

no force pushes. And probably not an intern.

But code that never gets looked at and only tested? Yes. And as AI gets better this will be happening in all companies... more and more.

[bigstrat2003]

Anyone who lets the LLM put out unreviewed code is shockingly negligent and bad at his job. It certainly might be the case that many people are bad in such a way, but that doesn't make it acceptable.

[threethirtytwo]

I accept it. You will too. Eventually. As AI improves, the reviews will just become more and more just time wasting blockers.

[jagenabler2]

How do you know your testing is complete if you're not sure what the underlying code does?

[threethirtytwo]

Testing will never be complete.

  def plus(a, b):
    return a + b

The above code, if a and b are integers have 2^64 test cases for full coverage. Code being much more complex then that on average will have a magnitude more test cases needed for full coverage... likely more test cases then all of the code humanity has ever wrote since the dawn of programming.

Testing is a statistical sample. You sample a small subset out of the full domain of possible tests and that sample you hope is somewhat representative of the actual population of tests. To put it in a sentence: If my suite of tests is correct then it is more likely that the entire full universe of every single possible test for my program can all be correct.

Realistically though the universe of all possible tests is so large that it's basically impossible for any sample to be a good representation of it. A suite of test does help up confidence that things will be MORE correct, but the sample size is so small it is almost guaranteed that you missed something.