[Bartweiss]

> the expectation of fallibility with human docs

Not just the expectation but the understanding. A doctor might very well forget which leg to amputate, so we know to Sharpie "NOT THIS LEG" on the one being kept. But a doctor is very unlikely to see a patient with a broken wrist and prescribe antipsychotics, so we don't do much to prevent that error. Human fallibility happens along fairly predictable channels, and we've spent a very long time committing resources to controlling those channels.

Watson, though, thought Toronto is a city in the USA. Anyone who's dealt with ML output knows that the errors are often quite surprising, even before dealing with adversarial inputs. Even in a system where Watson's outputs are subject to checks, the checks we have today are human-specific and developed at a significant human cost. ML answers can't just outperform individual human doctors to add value, they need to either be gracefully integrated with them or be able to outperform the entire system which keeps those doctors on track.