[esquire_900]

You are right. In that sense, the AI that puts out the best bullshit will eventually "win", as in general with humanity.

Your perspective (and most readers here) is very different then those of the general population though. The creator of an AI has a deep understanding of how it came to live (even if it does not fully grasp the current learned decision making). But for the average Joe, it's impossible to correctly decide to trust an AI or not.

So even though you're completely right, it's the direction we'll go in anyway. Either humans or other AI's vouch for an algorithm, and 99% of the population will just go with it. It's crap stacked on other crap, and some entities in the right place hugely abuse their position, but it's the only way.