[fabian2k]

AI can be quite impressive if the conditions are right for it. But it still fails at so many common things for me that I'm not sure if it's actually saving me time overall.

I just tried earlier today to get Copilot to make a simple refactor across ~30-40 files. Essentially changing one constructor parameter in all derived classes from a common base class and adding an import statement. In the end it managed ~80% of the job, but only after messing it up entirely first (waiting a few minutes), then asking again after 5 minutes of waiting if it really should do the thing and then missing a bunch of classes and randomly removing about 5 parenthesis from the files it edited.

Just one anecdote, but my experiences so far have been that the results vary dramatically and that AI is mostly useless in many of the situations I've tried to use it.

[manishsharan]

This is exactly my experience. We wanted to modernize a java codebase by removing java JNDI global variables. This is a simple though tedious task. And we tried Claude Code and Gemini. Both of these results were hilarious.

[Workaccount2]

LLMs are awful at tedious tasks. Usually because it involves massive context.

You will have much more success if you can compartmentalize and use new LLM instances as often as possible.

[benregenspan]

One thing I like for this type of refactoring scenario is asking it to write a codemod (which you can of course do yourself but there's a learning curve). Faster result that takes advantage of a deterministic tool.