[verdverm]

There is a belief that everyone is just taking whatever the LLM (really agents now) outputs. This is not the case anywhere I work. We use human oversight to have it iteratively improve the code. The average quality is going up.

[TheRoque]

Do you have some metrics to back this up ? Because from what I can see with my own eyes, outages everywhere, security holes everywhere too, doesn't seem that things are improving..

[verdverm]

There are insufficient metrics for both directions, and while you can find evidence to support either position, this discounts two things

1. Things are evolving. The models, and especially the harnesses, are getting better. There was an inflection point at the end of last year, so anything before that is no longer relevant to the discussion. Probably anything before now, since we've had about 6 months with real agentic engineering and things are starting to become clearer.

2. Application and effectiveness is not equally distributed. This is the newest and most significant technology humans have created. We are still building and figuring it out. Some people are better at it, some people use it rather unwisely.