[pardoned_turkey]

How does open source improve safety if we simply don't have the analytical tools to intuitively reason about LLMs?

You can't use this to prove that the model will always behave correctly (or desirably). At best, you can build test-suites to empirically check that it kinda-sorta appears to be doing the right thing most of the time. Which you can just as easily do with a black-box model.

It's not that I'm against openness. I just don't see how you can posit that it gets us close enough to safety.

[notahacker]

Full openness in healthcare also comes with a cost: no training or fine-tuning on patient records or real world interactions which really really ought to not be fully open.

[Espressosaurus]

Step one is transparency--let's get the black boxes under our control open.

It is not sufficient but it is necessary.

[pardoned_turkey]

Right, but the article doesn't make that point. It is full of magical thinking that openness is the one hurdle we need to clear.

I wouldn't feel any more comfortable getting diagnosed by an open-source LLM than I would be by a proprietary one made by OpenAI.

[pixl97]

But your fine with a human that might spend 5 minutes looking at your chart after a heavy day of drinking/pills/10,000 other things to do?

The problem with 'Doctor AI' isn't that it's going to make mistakes, we already have doctors doing that and killing 100s of thousand per year. It's that we'll only have 1 doctor AI everywhere and there won't be a thing as a second opinion because "Computer Don't Argue"