|
The problem is that the whole event was orchestrated to showcase IBM. Jeopardy didn't offer an open call. There's been no series of open competitions in Jeopardy-style trivia, as there was with gradually-improving chess computers. Instead, IBM wanted a forum to show off its multi-million-dollar QA technology, and approached Jeopardy. (They may have also, though I haven't seen definitive information either way, offered Jeopardy promotional payments.) IBM then spent 3+ years optimizing for the Jeopardy domain. (In the Reddit QA, the Watson team answered: "At this point, all Watson can do is play Jeopardy and provide responses in the Jeopardy format.") And in the matches, Watson dominated on one dimension of Jeopardy play – quickly pressing a button after a light goes off – that's the least interesting technical challenge. (Yes, it's an important part of any champion's skills, but a machine would have won that button-pressing competition 50 years ago, so it obscures rather than highlights any other 'breakthroughs' Watson may represent.) While impressive in several dimensions, and drawn from much deeper research by IBM, the only thing we can say for sure about Watson is that it was a "Horse for the Course" in Jeopardy. And unfortunately, no other computer horses were invited to play, and offered the same prizes (in money and fame). I suspect, now that the pattern has been set, we'll see leaner teams showing they can do as well or better than Watson with far less funding/hardware, over the next few years. Still, in the popular imagination, these efforts will live in the shadow of Watson, when a fair competitive process might have given them a chance to upstage Watson. |
Agree on your suspicion. Simply quartering the cost of memory and copying the approach from the paper with some home-grown improvements will get people ahead of IBM and probably inside IBM's decision loop so they're permanently ahead. But plowing something the first time is often the hardest. These weren't dumb people working on this thing for 3+ years.