[patrickhogan1]

Awesome! This is great!

The link in the article to the full blog explaining rerankers is 404ing for me.

Questions to you as an expert related to search ranking. With o3 and source quality thresholds when performing web search. Could we implement an ELO-style cutoff where systems default to “I don’t know” rather than citing low-ranked sources?

Currently o3’s main weakness is mixing high-quality sources with poor ones when it uses the web search in the same response. The answer sounds authoritative throughout, but parts are backed by unreliable sources. This makes it harder to trust even the well-sourced portions (e.g. believing the US election is next year - not a hallucination but a poorly date formatted source it used). It also makes the response a lot slower.

Would a hard quality threshold be better than the current approach of seamlessly blending good and bad sources?

[ghita_]

Hey! Thanks so much! I fixed the link thanks for flagging. Yes the same approach could be used for internet search. The fact that we now have an "absolute score" is very interesting since we can also use a threshold value to determine when an answer simply doesn't exist in a corpus. The only issue is that if all scores are below the cutoff value, you end up discarding them all, and end up with many "I don't know"s. Best approach could just be to flag the "trust" the model has in each source retrieved and use it as such.