I would expect that Watson is used as a discovery, not a decision tool - something to point out the symptoms as matching an uncommon disease that the doctor had not otherwise considered.
There are around 500 000 biology and medical research papers published per year (http://www.stm-assoc.org/2012_12_11_STM_Report_2012.pdf), and it's completely impossible for a doctor to keep up with all of the latest knowledge. A search engine can help a lot in discovering otherwise unnoticed connections.
I'm not sure how insurance companies are going to approach this - but "the computer agrees with my decision" seems to be a good defense argument in a malpractice case. And if the end result is statistically better, then insurance companies will adjust their prices.
Kaiser makes use of an expert system when determining whether to schedule an appointment. I once called in with the the symptoms, "Excruciating pain in the chest, left arm just went completely numb at the same time" - and the admitting nurse (under the guidance of an expert system) determined with three quick questions that it wasn't urgent. (Any shortness of breath, are you light headed, if you press your fingernail down, how long does it take to return from white to red)
So, expert systems are already here and helping make decisions...
With that history you should have been seen urgently. There are a number of serious things that are possible with those symptoms.
Plus, the nurse was relying on your accurate reporting of symptoms. Things are often very different face to face than over a telephone. People very often under/overplay their symptoms.
Presumably those people who've developed the kaiser system have come to a different conclusion than you. It may be the case that there is no scenario in which "Excrutiating Chest Pain + Numb Left Arm" is urgent when you have a full return to red on when pressing your fingernail (blood pressure check?)
Also, presumably, if I was having any shortness of breath, it could be determined just by talking to me. They may have also taken my age into account (32) and decided that 32 year olds don't have heart attacks in the way I was describing.
I came in for a checkup a week later, and all was well (except for a RSI issue that was contributing to the completely numb left arm)
Oh yeah this would never be purely AI...think of Watson like the Tardis and the Doctor like ...The Doctor. They go hand in hand, Watson simply makes it so the Doctor doesn't have to waste time.
Assuming it was more reliable, of course.