Hacker News new | ask | show | jobs
by 8note 3 days ago
how do you know that it is actually suggesting the right thing?
3 comments

Not OP, but: I guess in a similar fashion to when I google things or read other websites: I don’t, but I use my instinct, judgement, experience…

Very often I do catch LLMs, even the best such as Opus, confidently saying wrong things about areas in theory I know little of. And sometimes I fail to catch them and only realize that later on….sort of like…how I learned my whole career? So many wrong abstractions, tools, and so many hard earned lessons. With LLMs it’s the same, but the process is much faster. For critical decisions I don’t blindly trust an LLM, for example.

Some things are verifiable. Before coding agents, if I encountered an issue with a library or a framework, my first hunch would be to find a GitHub issue with a suggested workaround. Nowadays, I can ask an agent to really dig into it and often it does surface the root cause. For example, the other day I got a test hangup after updating to Angular 22, and the agent managed to find the bug and suggest a very trivial workaround compared to what I originally planned to go with. I reported the issue and it was fixed the next day, more or less along the lines of what I'd do.
I trust AI to surface general information and best practices on established knowledge domains. For example: best practices for securing my VPS.

For domains whete SoTA is constantly changing like AI, I use LLMs to aggregate and interact with my own research from trusted sources ala Karpathy LLM wiki.

I don’t generally trust everything I read on the internet whether its AI generated or not. I do my own research for the things that matter to me.