[alpha_squared]

> AI can (theoretically) be a lot more fair in dealing with claims

Respectfully, no it can't. From a Western perspective, specifically American, and from an average middle-class person's perspective, specifically American, it only appears to be fair.

However, LLMs are a codification of internet and written content, largely by English speakers for English speakers. There are <400m people in the US and ~8b in the world. The bias tilt is insane. At the margins, weird things happen that you would be otherwise oblivious to unless you yourself come from the margins.

[throw10920]

I don't even think most Americans (except those trying to do the automating) would consider it to be fair.

AI is bias automation, and reflects the data it's trained on. The vast majority of training data is biased, even against different slices of Americans. The resulting AI will be biased.

[Muromec]

On the other hand, once the claim is mishandled by AI, one can use the normal process to discover the juiced prompt and all the papertrail that comes with implementing it.

[nothercastle]

Yeah good luck getting the company to cough up what they call “Source code” during discovery

[bcyn]

> LLMs are a codification of internet and written content

Only true for pre-trained foundational models without any domain-specific augmentations. A good AI tool in this space would be fine-tuned or have other mechanisms that overshadow the pre-training from internet content.

[nradov]

Nope. Claims adjudication LLMs aren't trained on random Internet content. If you're going to criticize then at least get your basic facts right.

[Zanfa]

Why wouldn't they be? LLMs need a lot of content for training and there's multiple orders of magnitude less to train on of if you limited it to insurance-specific content, so you'd probably get a really crappy LLM. And training from scratch is really expensive anyway.

At best they'll be using fine tuned enterprise OpenAI / Anthropic models, more likely a regular model with a custom prompt.

[nradov]

They use actual production claims data and policy documents for training. Not random garbage from the Internet.

[Zanfa]

That’s not how LLMs work.