[AlwaysRock]

When? I've found over the last 6 months or so there has been a drastic change in quality of results. I used to have a ton of hallucinations like that. It still happens, but it is significantly less common now.

I also get better results when using a prompt window built into and ide than something detached like the openai website.

[linsomniac]

AFAICT, OpenAI o3-mini-high, a couple days ago completely hallucinated functionality within systemd homectl that would mount a home directory from another machine using sshfs. It certainly wasn't available on Ubuntu 24.04, but searching the Internet I can't find those options documented anywhere.

Though, I do agree that hallucinations have dropped dramatically.

[AlwaysRock]

I am not an expert in systemd homect but I have found that adding, "Show documentation for any methods or built in functions used" helps when I'm working on anything obscure or that is less likely to have a lot of publically availability code (that the LLM was likely to be trained on).

That helps a lot whenever I am working off something I know will likely exist but just needs to be pulled from a company's documentation.

[linsomniac]

That did provide some more interesting results, seemingly getting closer to something that would actually work, but not quite. Perplexity, Claude, and ChatGPT all provided very different approaches. That addition to the prompt definitely did provide some useful additional information.