|
|
|
|
|
|
by jampekka
42 days ago
|
|
|
AA-Omniscience Index gives +100 for correct, 0 for "I don't know" and -100 for incorrect. For your scenario the confident confident strategy will give average of -90. Saying I dont't know to all will give 0. A lot of models have negative AA-Omniscience Index. They also do have AA-Omniscience Accuracy and AA-Omniscience Hallucination Rate that handle "I don't knows" differently. https://artificialanalysis.ai/evaluations/omniscience |
|
|