I am not sure if I'm ignorant or have too much self esteem, but in cases where domain is not my expertise I try to cross challenge using different models to get less-biased and hopefully more correct insights.
I’m not trying to be snarky, but.. I wouldn’t try to upgrade an answer from a stranger you can’t fully trust to an expert answer you can trust by asking more strangers who have all only read the same books and use the same system for summarizing the books.
Similarly, sometimes I have to manually validate the output of an ai tool, analyzed over a large text I can’t practically read and understand, and best way I’ve found is to ask the tool (or another ai) to ‘show your work,’ ie make it help me make the determination by showing places in the text I have to read to follow its reasoning. We can never trust another ai directly to assess the validity of another ai.
This is some work that I want to see in education. How do you validate if something is true or not.
A lot of times that will take physical tests. Or in the case of math/logic, tests to validate each line or validated sources of previously proved theorems
You use the scientific method. Make a guess about how the system works, do experiments aimed to show that's not the case, adjust guess accordingly, and repeat
This works only when the inputs and conditions are all the same. In the real world, that isn't really possible anymore for a lot of things, like data sciences operating within a large pulsating mass of live net.
Similarly, sometimes I have to manually validate the output of an ai tool, analyzed over a large text I can’t practically read and understand, and best way I’ve found is to ask the tool (or another ai) to ‘show your work,’ ie make it help me make the determination by showing places in the text I have to read to follow its reasoning. We can never trust another ai directly to assess the validity of another ai.