And whichever method you use, you’re still accountable to regulators, courts, the letter of your contract, and the consequences of your reputation in a competitive market.
United Healthcare was in the news last year because they had an AI claims "approval" process with a 90% error rate, all in favor of the insurance company.
It's easy to describe a business process with written down rules, and those are easy to find in legal discovery. It's much easier to obfuscate with an AI model, because "nobody knows what it's actually doing - it's AI!".
It was not a 90% error rate (or at least that’s not a claim I read). It was that 90% of appeals of those decisions were decided (at least partially) in favor of the appeal. That could be 1000 decisions, 10 appeals, and 9 reversals.
I am personally 7 for 8 in lifetime wins in my city's parking ticket appeals process. That doesn't mean that I think that 7 out of 8 tickets my city issues are incorrect.
> It's much easier to obfuscate with an AI model, because "nobody knows what it's actually doing - it's AI!".
Do you have actual knowledge of this? If not, the most obvious counterpoint is that the AI will need to give the reason or reasons for denial, and recording them for audit. Just like a human or a rules-based system.