[jamesbrady]

Elician here! Thanks for your comment.

I'm not sure I agree that those rule-of-thumb statistics are "arbitrary" or "fictional"… I guess it depends on what you mean by that. I can say that on our part they're a good faith attempt to help users calibrate how best to use the tool, using evaluations of Elicit based on real usage.

Definitely accept that the tool can work better or worse depending on your domain or workflow though!

One way we do try to distinguish ourselves from vanilla LLMs is that we provide sources for all of the claims made. I mention this because we hope our users can approach the falsification process you mention for Google. We want to show people where particular claims come from such that we earn their trust.

Walking citation trails and verifying transitive claims is something we've talked about but need more people to implement! (https://elicit.com/careers)

[nicklecompte]

> I'm not sure I agree that those rule-of-thumb statistics are "arbitrary" or "fictional"… I guess it depends on what you mean by that.

Sorry for the confusion: I meant that fragmede's comment was arbitrary and fictional, not the 90% figure. I was talking about these numbers:

  if it takes 1 hour to get one answer by hand, but only 20 minutes for the machine, and 20 minutes to check the answer, the user still comes out ahead

[jamesbrady]

Oh, my bad—I misunderstood, thanks for the clarification