[rjb7731]

Isn't this similar to the Deepmind paper on long form factuality posted a few days ago?

https://arxiv.org/abs/2403.18802

https://github.com/google-deepmind/long-form-factuality/tree...

[Xudong]

Yes, they are similar. Actually, our initial paper was presented around five months ago (https://arxiv.org/abs/2311.09000). Unfortunately, our paper isn't cited by the DeepMind paper, which you may see this discussion as an example: https://x.com/gregd_nlp/status/1773453723655696431

Compared with our initial version, we have mainly focused on its efficiency, with a 10X faster checking process without decreasing accuracy.

[westurner]

> We further construct an open-domain document-level factuality benchmark in three-level granularity: claim, sentence and document

A 2020 Meta paper [1] mentions FEVER [2], which was published in 2018.

[1] "Language models as fact checkers?" (2020) https://scholar.google.com/scholar?cites=3466959631133385664

[2] https://paperswithcode.com/dataset/fever

I've collected various ideas for publishing premises as linked data; "#StructuredPremises" "#nbmeta" https://www.google.com/search?q=%22structuredpremises%22

From "GenAI and erroneous medical references" https://news.ycombinator.com/item?id=39497333 :

>> Additional layers of these 'LLMs' could read the responses and determine whether their premises are valid and their logic is sound as necessary to support the presented conclusion(s), and then just suggest a different citation URL for the preceding text

> [...] "Find tests for this code"

> "Find citations for this bias"

From https://news.ycombinator.com/item?id=38353285 :

> "LLMs cannot find reasoning errors, but can correct them" https://news.ycombinator.com/item?id=38353285

> "Misalignment and [...]"