Thanks for trying it out - we're still quite early so the model isn't going to be perfect + we're focused on programming related queries at the moment. What is your use case? Are most of your searches related to history?
Most of my searches are related to researching and fact-checking potentially spurious statements, like the ones that Hello apparently produces.
This particular query about the French revolution seems to give GPT fits across its iterations, and I suspect it's because:
- the most correct answer is "I don't know"; when answering questions, GPT isn't really trying to answer a question by reasoning through it, it's trying to mimic what usually happens after someone asks a question, and especially online that usually isn't someone saying "I don't know"
- the most authoritative sources on subjects like this include books, and GPT doesn't seem to read a lot of those, or if it has then it doesn't seem to be able to cite them without inventing false details about the sources themselves (titles that don't exist, attributing works to the wrong authors)
Seeing "model that cites sources" made me excited that there was a solution to the second point. But specific to Hello:
- the most authoritative web sources are historians who cite sources, which aren't often written for SEO, on popular platforms, or not paywalled
- a search result being in a first page of Bing/Google doesn't make it authoritative
- this doesn't seem to stop GPT from coming up with false inventions/hallucinations that aren't in or relevant to the cited source at all
You're right - there are so many good sources not often surfaced by a cursory web search like books and experts. Adding better sources is something we're improving on. A simple next step we're looking into is to expand sources to research papers e.g explicitly pulling from arxiv on some queries.
This particular query about the French revolution seems to give GPT fits across its iterations, and I suspect it's because:
- the most correct answer is "I don't know"; when answering questions, GPT isn't really trying to answer a question by reasoning through it, it's trying to mimic what usually happens after someone asks a question, and especially online that usually isn't someone saying "I don't know"
- the most authoritative sources on subjects like this include books, and GPT doesn't seem to read a lot of those, or if it has then it doesn't seem to be able to cite them without inventing false details about the sources themselves (titles that don't exist, attributing works to the wrong authors)
Seeing "model that cites sources" made me excited that there was a solution to the second point. But specific to Hello:
- the most authoritative web sources are historians who cite sources, which aren't often written for SEO, on popular platforms, or not paywalled
- a search result being in a first page of Bing/Google doesn't make it authoritative
- this doesn't seem to stop GPT from coming up with false inventions/hallucinations that aren't in or relevant to the cited source at all