Hacker News new | ask | show | jobs
by sillyfluke 4 days ago
>No, but the same can be said for your colleagues.

That's absolutely false. My collegues don't routinely and confidently invent apis that are not there, or spectacularly and repeatedly misunderstand the purpose of certain functions or exhibit extreme forgetfullness. Especially when I've warned them. Hallucinations and confabulations in otherwise healthy individuals are mental disorders. When I ask them why they made an certain kind of error, I can expect to get a reasonable answer. No one has uttered the phrase "Bob hallucinated again while writing those tests" when the Bob in question is a human.
1 comments
deanc 4 days ago
Well, your experience doesn't align with mine. I have been using, and in part of an organisation that is extensively using, Claude with Opus for everything for about 3 months now and I am not experiencing the problems you describe. We'll have to agree to disagree here.
link
sillyfluke 4 days ago
That is fine. "Your experience may vary" is the crux of my argument amusingly. You can't have just realized that people are having different experiences using AI, or even that the same person has different experiences when they change domains or technical contexts. There's been lots of comments littered on this forum to that effect.

Calling hallucinations simply mistakes does not seem to me to be a healthy way to reason about LLMs. I can ask a collegue how well they can program in Ada and adjust my expectations on productivity and bug rates. I can't ask an LLM how well they can code in Ada (just a throwaway example), or even how much of Ada was in its training data. I have to actually spend money and spend time code reviewing before I can even formulate any expectations at all.

link
rfgplk 4 days ago
Not only have I never ran across a hallucination in the past ~6 months or so; the latest Opus models have gotten to the point where they can emit inline assembly that is _superior_ to what gcc or clang can generate from optimized cpp. Had it rewrite a hot simd loop that took it from ~10 flops/cyc to ~14 by shaving off broadcasts. I _could not_ get any compiler to do this, no matter which flags I tried to use. So I literally have no idea what these people are talking about when they claim that SOA models hallucinate constantly.
link
shakna 4 days ago
Last week, Opus gave me a decrement instead of an increment, on one particular line. Where I already had the decrement, but it was changing the width of the datatype everywhere.

And it took "convincing" that it had made a mistake.

link