[p_l]

I worked on a project where one of the services was a model that decided whether to pay a medical bill.

Before you start justified screams of horror, let me explain the simple honesty trick that ensured proper ethics, though I guess at cost of profit unacceptable to some corporations:

The model could only decide between auto approving a repayment, or refer the bill to existing human staff. The entire idea was that the obvious cases will be auto approved, and anything more complex would follow the existing practice.

[conartist6]

Mmmmhm, which means the humans now understand that they should be callous and cold. If they're not rubber stamping rejections all the time then the AI isn't doing anything useful by making a feed of easy-to-reject applications.

The system will become evil even if it has humans in it because they have been given no power to resist the incentives

[rglullis]

> humans now understand that they should be callous and cold

Were humans working on health insurance claims previously known for being warm and tend to err on the side of the patient?

[ethbr1]

> Were humans working on health insurance claims previously known for being warm and tend to err on the side of the patient?

I know that in the continuously audited FEP space, human claims processors were at 95%+ accuracy (vs audited correct results).

Often with sub-2 min per claim processing times.

The irony is that GP's system is exactly how you would want this deployed into production. Fail safe, automate happy path, HITL on everything else.

With the net result that those people can spend longer looking at more difficult claims. (For the same cost)

[lesuorac]

All you have to do is take an initial cost hit where you have multiple support staff review a case as a calibration phase and generate cohorts of say 3 reviews where 2 have the desired denial rate and 1 doesn't. Determine the performance of each cohort by how much in agreement they are and then rotate out whose in training over time and you'll achieve a target denial rate.

There will always be people who "try to do their best" and actually read the case and decide accordingly. But you can drown them out with malleable people who come to understand if they deny 100 cases today then they're getting a cash bonus for alignment (with the other guy mashing deny 100 times).

Technology solves technological problems. It does not solve societal ones.

[rglullis]

I am not disagreeing, and I am not arguing for AI.

I am just saying that the perverse incentives already exist and that in this case AI-assisted evaluation (which defers to a human when uncertain) is not going to make it any better, but it is not going to make it any worse.

[ToucanLoucan]

Actually it may, even if only slightly. Because now as the GP says, the humans know the only cases they're going to get are the ones the AI suspects are not worthy. They will look more skeptically.

I totally agree that the injustices at play here are already long baked in and this is not the harbinger of doom, medical billing already sucks immense amounts of ass and this isn't changing it much? But it is changing it and worse, it's infusing the credibility of automation, even in a small way, into a system. "Our decisions are better because a computer made them" which doesn't deal at all with how we don't fully understand how these systems work or what their reasoning is for any particular claim.

Insofar as we must have profit-generating investment funds masquerading as healthcare providers, I don't think it's asking a ton that they be made to continue employing people to handle claims, and customer service for that matter. They're already some of the most profitable corporations on the planet, are costs really needing cutting here?

[rglullis]

>"Our decisions are better because a computer made them"

This is the root of the problem, and it is (relatively) easy to solve: make any decision taken by the computer directly attributed to the CEO. Let them have some Skin in The Game, it should be more than enough to align the risk and the rewards.

[ballenf]

The bot should have let ~5% of auto-accepted claims through to the humans. And then tracked their decision.

[p_l]

Actually the real issue for the humans was that it would mean possible reduction in employment which is why we had union block deployment for a time until a deal was brokered.

It helps, as you can suspect from "union" comment, that it wasn't an american health care insurance company.

[Eddy_Viscosity2]

How hard would it be tweak that model so that it decides between auto-paying and sending it to a different bot that hallucinates reasons to deny the claim? Eventually some super smart MBA will propose this innovative AI-first strategy that will boost profits.

[supplied_demand]

Funny enough, the large AI companies run by CEOs with MBAs (Alphabet and MSFT), seem to be slow-playing AI. The ones promising the most (Meta, Tesla, OpenAI, Nvidia) are led by strict technologists.

Maybe it’s time to adjust your internal “MBAs are evil” bias for something more dynamic.

[chrisweekly]

In what way is MSFT "slow-playing" AI?

[supplied_demand]

They are slow-playing the promise of what AI can, should, and will accomplish for us.

Nadella said this yesterday at YC’s AI Startup School:

== “The real test of AI,” Nadella said, “is whether it can help solve everyday problems — like making healthcare, education, and paperwork faster and more efficient.”

“If you’re going to use energy, you better have social permission to use it,” he said. “We just can’t consume energy unless we are creating social and economic value.”==

https://www.thehansindia.com/tech/satya-nadella-urges-ai-to-...

[chrisweekly]

Thanks. I agree w the things Nadella said there. But it rings pretty hollow, given how hard every MSFT product is pushing AI. What would it look like if they weren't "slow-playing" it?

[supplied_demand]

That’s fair. I was looking more at the promises of what it can/will do than integrating it into products. The MBA CEOs seem more focused on solving business problems and the tech CEOs are more focused on changing the world.

This is all an aside from the original point, which was that I think it is unfair to pin the proliferation and promises made about AI on some cabal of MBAs somehow forcing it. The people building the tools are just as at fault, if not more.

[bird0861]

Right, I can't sustain for a moment the idea that the guy who fumbled Recall like a stack of wet fish dipped in baby oil is actually a wise sage full of caution. I permit myself one foolish idea a day and that's not going to be the one for any day of the week.

[Eddy_Viscosity2]

Indeed, it is often the case that what powerful people say is very different from what they do.

[thfuran]

Harder than just automatically rejecting every claim.