[pyman]

I used GPT-5 as my personal lawyer the other day. I uploaded contracts and agreements, and the number of things it got wrong was mind blowing. It made me look like a complete amateur with the emails it drafted. GPT-5 is extremely dumb and incompetent when it comes to understanding real-world problems and offering solutions. It wasn't even at the level of a junior lawyer, it was ten times worse. I was honestly shocked by the results.

When it comes to programming, I have to keep replaying with "Nope", "No", "Again", "Wrong", "It doesn't work" and a couple of times "Do better" before it finally produces something complex that actually works.

With coding at least I know when it's wrong. The real problem is when I don't.

[ohr]

But if you let it run the tests, it’ll just do that on its own (or delete the tests, but usually it won’t), which is very useful and seems to match the article’s sentiment.

[pyman]

Test-driven development is back.

Since LLMs rely on patterns learned from large amounts of text, the results are relative. If the training data contained more Rust repos, that could explain why it feels stronger in Rust.

The way AI companies talk about "intelligence" now is shifting. They admit LLMs can't truly reason with the current architecture, so intelligence is being framed as the ability to solve problems using patterns learned from text, not reasoning on their own. That's a big downgrade from the original idea of AI reaching human-level thinking and developing AGI.

Also, my understanding is that since Microsoft invests in Copilot, it doesn't want ChatGPT to get better at coding. Instead, it wants it to get better at being a lawyer.