[geysersam]

They do tend to overengineer. The other day 5.6 Sol generated a while loop around a uuid4 call to make sure the generated ids were unique...

I wonder if that's also how I'd behave if I had a reinforcement learning harness around me that dived hard on and punished me for every small mistake.

[denverllc]

That is not overengineering, it’s creating job security:

> Funny story no one will believe, but it’s true. A good friend of mine joined a startup as CTO 10 years ago, high growth phase, maybe 200 devs… In his first week he discovered the company had a microservice for generating new UUIDs. One endpoint with its own dedicated team of 3 engineers …including a database guy (the plot thickens). Other teams were instructed to call this service every time they needed a new ‘safe’ UUID. My pal asked wtf. It turned out this service had its own DB to store every previously issued UUID. Requests were handled as follows: it would generate a UUID, then ‘validate’ it by checking its own database to ensure the newly generated UUID didn’t match any previously generated UUIDs, then insert it, then return it to the client. Peace of mind I guess. The team had its own kanban board and sprints.

https://news.ycombinator.com/item?id=48061235

[layer8]

> They do tend to overengineer. The other day 5.6 Sol generated a while loop around a uuid4 call to make sure the generated ids were unique...

That’s not overengineering, it’s plain nonsensical, because presumably it doesn’t compare it to all IDs generated in the past. Which, if you wanted to do that, you’d use a database with a uniqueness constraint, in case you don’t already have that anyway.

Fixing this lack of reliable common-sense awareness seems to remain elusive for LLMs.

[geysersam]

The ids were used in a limited context and it compared to all the other ids generated in the same context. Yes, absolutely nonsensical.

I find it interesting that presumably the LLM was trained on millions of legit examples where people generate uuids, but still it decided to do something different and unexpected, because it "reasoned" itself into thinking it was necessary.