[ummonk]

“LLMs can't to my knowledge process a request into a lookup on say an actual database of facts at the moment or parse a request into API actions.”

Both Bing chat and ChatGPT plugins are examples of being able to do just this.

You’re right about how they make up answers though, but humans are often quite prone to that too…

[rtkwe]

A human, if not incentivized to lie or directly incentivized to be truthful, could at least tell you when they're making something up themselves where Bing/Bard seemingly cannot. Once it can do that I think they'll be far more useful, at least then you can have a rough idea of how much you need to check the bots work. If I have to do that for every thing it spits out the best it can do for me is give me new words to use while searching.

Granted getting the name for something to search is often half the battle in tech.

[smallnamespace]

> could at least tell you when they're making something up themselves where Bing/Bard seemingly cannot.

In fact GPT-4 is quite good at catching hallucinations when the question-answer pair is fed back to itself.

This isn’t automatically applied already because the model is expensive to run, but you can just do it yourself (or automate it with a plug-in or LangChain) and pay the extra cost.

Remember that the model only performs a fixed amount of computation per generated token, so just asking it to think out loud or evaluate its own responses is basically giving it a scratchpad to think harder about your question.