The approach of generating something and then looking for hallucinations is just stupid. To validate the output I have to be an expert. How do I become an expert if rely on LLMs? It's a dead end.
"and suport the claims" is doing some *extremely* heavy lifting there.
I can't write a software program, give the source to the greengrocer and expect him to be able to say anything about its quality. Just like I can't really say much about vegetables.
If the output is interpreting sources rather than just regurgitating quotes from them, you need to exert judgment to verify they support its claims. When the LLM output is about some highly technical subject, it can require expert knowledge just to judge whether the source supports the claims.
No. You only need to check for sources, and then verify these sources exist and they support the claims.
It's the very definition of "fact".
In some cases, all you need to do is check if a URL that was cited does exist.