[southernplaces7]

One reads completely ridiculous cases like the one you describe, and shakes their head at those who preach the notion of creating ever more thickets of AI "powered" bots as a prima facie interface for our social services, customer support and other institutional interaction needs.

Idiocies like this are why AI should absolutely never (at least at any present level of technology) be an inescapable means of filtering how a human is responded to with any complaint. Truly, fuck the mentality of those who want to cram this tendency down the public's throat. Though it sadly won't happen thanks to sheer corporate growth inertia, companies that do push such things should be punished into oblivion by the market.

[p_l]

I worked on a project where one of the services was a model that decided whether to pay a medical bill.

Before you start justified screams of horror, let me explain the simple honesty trick that ensured proper ethics, though I guess at cost of profit unacceptable to some corporations:

The model could only decide between auto approving a repayment, or refer the bill to existing human staff. The entire idea was that the obvious cases will be auto approved, and anything more complex would follow the existing practice.

[conartist6]

Mmmmhm, which means the humans now understand that they should be callous and cold. If they're not rubber stamping rejections all the time then the AI isn't doing anything useful by making a feed of easy-to-reject applications.

The system will become evil even if it has humans in it because they have been given no power to resist the incentives

[rglullis]

> humans now understand that they should be callous and cold

Were humans working on health insurance claims previously known for being warm and tend to err on the side of the patient?

[ethbr1]

> Were humans working on health insurance claims previously known for being warm and tend to err on the side of the patient?

I know that in the continuously audited FEP space, human claims processors were at 95%+ accuracy (vs audited correct results).

Often with sub-2 min per claim processing times.

The irony is that GP's system is exactly how you would want this deployed into production. Fail safe, automate happy path, HITL on everything else.

With the net result that those people can spend longer looking at more difficult claims. (For the same cost)

[lesuorac]

All you have to do is take an initial cost hit where you have multiple support staff review a case as a calibration phase and generate cohorts of say 3 reviews where 2 have the desired denial rate and 1 doesn't. Determine the performance of each cohort by how much in agreement they are and then rotate out whose in training over time and you'll achieve a target denial rate.

There will always be people who "try to do their best" and actually read the case and decide accordingly. But you can drown them out with malleable people who come to understand if they deny 100 cases today then they're getting a cash bonus for alignment (with the other guy mashing deny 100 times).

Technology solves technological problems. It does not solve societal ones.

[rglullis]

I am not disagreeing, and I am not arguing for AI.

I am just saying that the perverse incentives already exist and that in this case AI-assisted evaluation (which defers to a human when uncertain) is not going to make it any better, but it is not going to make it any worse.

[ToucanLoucan]

Actually it may, even if only slightly. Because now as the GP says, the humans know the only cases they're going to get are the ones the AI suspects are not worthy. They will look more skeptically.

I totally agree that the injustices at play here are already long baked in and this is not the harbinger of doom, medical billing already sucks immense amounts of ass and this isn't changing it much? But it is changing it and worse, it's infusing the credibility of automation, even in a small way, into a system. "Our decisions are better because a computer made them" which doesn't deal at all with how we don't fully understand how these systems work or what their reasoning is for any particular claim.

Insofar as we must have profit-generating investment funds masquerading as healthcare providers, I don't think it's asking a ton that they be made to continue employing people to handle claims, and customer service for that matter. They're already some of the most profitable corporations on the planet, are costs really needing cutting here?

[ballenf]

The bot should have let ~5% of auto-accepted claims through to the humans. And then tracked their decision.

[p_l]

Actually the real issue for the humans was that it would mean possible reduction in employment which is why we had union block deployment for a time until a deal was brokered.

It helps, as you can suspect from "union" comment, that it wasn't an american health care insurance company.

[Eddy_Viscosity2]

How hard would it be tweak that model so that it decides between auto-paying and sending it to a different bot that hallucinates reasons to deny the claim? Eventually some super smart MBA will propose this innovative AI-first strategy that will boost profits.

[supplied_demand]

Funny enough, the large AI companies run by CEOs with MBAs (Alphabet and MSFT), seem to be slow-playing AI. The ones promising the most (Meta, Tesla, OpenAI, Nvidia) are led by strict technologists.

Maybe it’s time to adjust your internal “MBAs are evil” bias for something more dynamic.

[chrisweekly]

In what way is MSFT "slow-playing" AI?

[supplied_demand]

They are slow-playing the promise of what AI can, should, and will accomplish for us.

Nadella said this yesterday at YC’s AI Startup School:

== “The real test of AI,” Nadella said, “is whether it can help solve everyday problems — like making healthcare, education, and paperwork faster and more efficient.”

“If you’re going to use energy, you better have social permission to use it,” he said. “We just can’t consume energy unless we are creating social and economic value.”==

https://www.thehansindia.com/tech/satya-nadella-urges-ai-to-...

[chrisweekly]

Thanks. I agree w the things Nadella said there. But it rings pretty hollow, given how hard every MSFT product is pushing AI. What would it look like if they weren't "slow-playing" it?

[thfuran]

Harder than just automatically rejecting every claim.

[molteanu]

Nice that you're mentioning it. I've seen this piece today from Bloomberg, "Call Center Workers Are Tired of Being Mistaken for AI."

https://archive.ph/rB2Rg