[Tadpole9181]

I disagree, this is all because it's understood.

You need a system promot because an LLM is fundamentally a token predictor. It needs to be primed for the work it's going to do, otherwise it's next token prediction has too little to go off in the beginning and goes nuts.

You use a separate AI to review the code, because it's a token predictor trained mostly on accomplishing tasks in a cost effective way - we haven't made AGI here. The first shot that actually makes the code will have rationalizations in its memory as well as prior research, which bias token prediction to accepting that as true (remember how they had to train our sycophancy?). Then because of the desire to be cost effective, they're slightly lazy, and so it won't always do the research to find new edge cases and problems and missing tests the initial research didn't find.

So spin up a separate, clean slate, and ask for it to review from scratch. Or have that one spin up multiple smaller ones to have them specialize in specific concerns or domains (security, data model corruption, code quality), then have the orchestrator validate those concerns and stitch them into a cohesive response.

I can't help you on your last question. Saying they are degrading in quality is not even close to my experience. 5.6 and Opus/Fable 5 are have been a huge step up. Though I do need to adjust memory rules as these new models come out, since ground-up retrained models often come with their own quirks that replace old ones - causing old, specific memories to have unintended side effects.

[seff]

Yea, it's like people don't understand that it's a probability machine. We're steering it to the best probable outcome.

It's not magic, it's math.

[throwatdem12311]

My point is that I understand all of that but in my view that makes them fundamentaly flawed at a conceptual level for what people are saying they should be used for.

[Tadpole9181]

Why? I'm an "artisanal programmer" at heart, but for the past year I've been using these to great effect. A car that needs to be started with a hand-crank and occasionally stalls is still a car.

[CoolestBeans]

An artisan who uses a mass production factory to make their goods isn't really an artisan.

[Tadpole9181]

I mean, insult me all you want, but you'd be pissed if your mechanic didn't use code readers or your carpenter used hand drills to pad their hours.

I do hand programming for myself now and use the right tools in my office, because I'm a responsible adult who understands that I am not paid to have fun or feel special and smart - it's to produce a product for my employer.

[throwatdem12311]

I’d be pissed if my mechanic wasn’t using code readers because code readers are forced on them by manufacturers, that doesn’t mean I wouldnt prefer a car that doesn’t need code readers at all.