Hacker News new | ask | show | jobs
by brigadier132 660 days ago
You are getting downvoted but you are right, a typo in a variable that already exists in a file like this is not the failure mode for LLMs. The failure mode is logic bugs, making up methods / functions.
1 comments

No, you can get variable names "mutated" on follow up requests. The thing is like sculpting with toothpaste.
I've been using copilot for as long as it has existed and what you are describing has not happened to me once. Literally on in the background 8 hours a day. Excuse me for not trusting the internet hivemind that hates everything that is hyped just a little bit.
My goto check of AI assistants is asking to write a function calculating the first N digits of Pi in Common Lisp. On at least two attempts when prompted to fix its code the model would change one of the variable names to T, which is a reserved symbol. So yeah pretty sure it does happen.