Web Concepts

Posts Tagged ‘jeopardy’

What is...Toronto (General Hospital)?

When Watson was competing on Jeopardy!, its massive databanks were filled with encyclopedias, novels, film scripts, and history books. These days, Watson is more into medical journals and misspelled Yahoo Answers blog posts about weird rashes and vague abdominal pains. Watson is maturing, and prepping for his first non-trivia, real-world application: medical diagnoses. He's all *sniff* grown up!

We've known medicine was to be the next step for Watson for some time, but just recently, IBM gave a short demonstration of Watson's progress. Watson isn't the first attempt at an automated diagnosis program--we documented Artemis, and Isabel has been around for a few years--but Watson's incredible power, depth of knowledge, and ability to understand natural human language puts it in a totally different league. Diagnosing an ailment isn't really that much different from answering a trivia question; Watson takes in as much information as possible from the question, eliminating the potential answers as new information renders them impossible, and comes up with a list of likely answers. An example from the AP: "As more clues were unveiled - blurred vision, family history of arthritis, Connecticut residence - Watson's suggested diagnoses evolved from uveitis to Behcet's disease to Lyme disease. It gave the final diagnosis a 73 percent confidence rating."

While on Jeopardy!, Watson could only give one answer, but in medicine, it lists all possible answers, along with the percent likeliness. An 80% possibility of accuracy is enough for Watson to risk money on Jeopardy, but when working with possible diagnoses, that still leaves a one in five chance that the patient is afflicted with something else, so Watson is designed to divulge even the less likely answers.

Watson's human language recognition skills also allows it to input an entirely new sector of information: anecdotal evidence. Anecdotal evidence is not necessarily reliable, of course, but can still be extremely useful--it's worth noting that a patient's description of symptoms is anecdotal, and still very important to diagnosis. Watson is able to trawl through the internet, picking up the oodles of medical information out there and adding it to its memory banks. Being able to understand that, say, a "dry mouth" is the same as xerostomia can make legitimate use of all those confused forums.

Of course, Watson isn't designed to replace a doctor's diagnostic instincts. Instead, it's more like a futuristic reference book. There's simply too much information out there these days, in too many places and added too frequently, for any doctor to keep up. Watson could help keep track of all the new drugs, studies, journals, and anecdotal evidence.

Diagnosis systems using Watson are still likely a few years away, but IBM is working on ways to leverage Watson's abilities even to hospitals with budgets too small to afford a multimillion-dollar Watson of their own. iPad apps were mentioned as a distinct possibility--doctors could tap into an off-site Watson with an iPad, shoot off a few queries and receive an answer immediately. And as more medical data is digitized, Watson will only get stronger and more useful.

[Associated Press]

Hey, it can win on Jeopardy, so let's put our lives in its hands

"Absolutely!" In Ferrucci's example, say Watson reads somewhere that "cytoplasm is a fluid that cushions the nucleus." Sharp Watson takes this information with a grain of salt, since it is uncorroborated. Later, Watson is told authoritatively that "cytoplasm is a liquid that surrounds the nucleus." At this point, Watson has burning questions to investigate, Ferrucci explains. "Is a liquid a fluid? Does surrounding mean cushioning?"

Awesome. Spoon-feeding information to a giant brain is not really a scaleable process, so I'm glad to know that the machine is using its skills to teach itself about the world, rather than being dependent on fallible meatbrains for deciding what it should learn next.

Meanwhile, having won on Jeopardy, Watson is getting a job in a hospital.

After the airing of the latest triumph (and crowning as a "new computer overlord" by trivia-master Ken Jennings [read his side of the story at Slate]), IBM announced Watson's next project: a foray into healthcare. IBM will be working with Nuance Communications, a company that works on speech recognition for medical diagnoses, as well as physicians at both Columbia University and the University of Maryland. They'll be working on using Watson's analytics in a way that'll help doctors make faster and smarter diagnoses--Watson is able to quickly scan mountains of information, from journals to prior cases and all kinds of other medical literature, finding relevant facts much faster than humans could. A one-word indicator in a patient's description of a symptom could trigger Watson to find applicable data, which he can do instantly, which a doctor could miss.

IBM and Nuance have a multi-year research project in the works, but expect to have some sort of commercial project ready within 18-24 months.

AI in prime time

Today at 7PM Eastern, the three-day mini-tournament will kick off, pitting IBM's supercomputer Watson against the two greatest modern Jeopardy! champions: Ken Jennings, who holds the record for longest winning streak, and Brad Rutter, who has taken home the most total prize money. It's an epic battle we've been looking forward to ever since we say Watson take on the champions in that preview battle last month--and it's finally here. But actual trivia knowledge may not turn out to be the deciding factor in the contest.

Over at Boing Boing, former champion Bob Harris (the 13-game Jeopardy! winner, not to be confused with the Lost in Translation character) lays out perhaps the most overlooked variable in Jeopardy!: the buzzer. We've been concentrating largely on the task of actually figuring out the answers to the questions (or questions to the answers--thanks for that weird rule, Jeopardy!), which for the layman is certainly challenging. The mix of trivia, riddles, puzzles, irony, and wordplay were assumed to be the toughest part of the game, but Harris lets us in on a secret: It's all about the buzzer. As he says, "At the top tournament level, every player can figure out nearly all of the correct responses, no matter how arcane."

When Trebek finishes reading a clue, a producer hits a button that flashes a light, telling the contestants the buzzers can now be triggered. Once you're at this top echelon of trivia mastery, the questions aren't the deciding factor--it's all about timing. Harris notes that "Since a computer can obviously react to the 'go' lights more rapidly and consistently than any human, it will probably win," providing Watson is allowed to buzz in as fast as possible. That corroborates what I saw in the practice match: None of the contestants, man or machine, actually got a clue wrong, and none went unanswered. This game may turn into a speed contest.

But that doesn't mean it won't be incredibly fun (and romantic) to watch.

Can a computer beat a human in the most challenging trivia game on TV? Today, at IBM's headquarters in New York, we learned that the answer is yes

Watson, named after IBM's founder, is one epic supercomputer. To handle the formidable task that competing on Jeopardy! presents, IBM spent years constructing a computer with 2,800 Power7 cores. That power is absolutely necessary--a single-core CPU, like in many modern computers, takes about two hours to come up with an answer to a standard Jeopardy! question, rather than the three-second average Watson currently boasts.

A lot of the challenge in creating an algorithm that can answer Jeopardy! questions lies in the questions themselves--the language used in these questions is hardly ever simple, often incorporating wordplay, riddles, and irony--but there's an additional problem in the addition of risk. In a split-second, a competitor must assess confidence in the question, weigh that confidence against the penalty of getting it wrong, and decide if the question is worth answering based on those factors. That's an intuitive effort for a human, but Watson had to be programmed with some incredibly complex reasoning to be able to do the same thing.

Watson has a certain self-awareness; it knows it won't get every answer right, and has to pass a certain level of confidence before it will answer. Watson's logo will change color to indicate its confidence: The lines that are part of its "avatar" will glow blue if Watson is confident, and orange if it's not.

The vagaries of language mean that the questions can be interpreted in all kinds of different ways, so merely figuring out what the question is trying to ask provides the majority of the struggle for Watson. To that end, the computer actually comes up with thousands of different possible answers, and ranks them by the possibility of correctness. When we watched the quick match, the top three answers were displayed on screen, as well as the confidence percentage, and the second- and third-ranked answers were usually dramatically incorrect. It's not likely that Watson will confuse, say, the author of one children's book with the author of another. It's more likely that Watson will completely misread what the question is even asking, and come up with an answer like "What is children?"

In this introductory battle, we learned a few things about the adjustments made to the show to accommodate a more mechanical being than usual. The question feed goes directly into Watson, so it doesn't have to "read" the question like the human competitors. But Watson does have to press a physical button to ring in, just like Ken Jennings and Brad Rutter, which pretty much eliminates the split-second advantage the computer has.

Interestingly, Watson will not be connected to the Internet, so there won't be any instant Wikipedia lookups. (IBM's reasoning: "Ken [Jennings] and Brad [Rutter] aren't connected to the Internet, so Watson shouldn't be either.") So where does this AI brain get its information? IBM's engineers, without the benefit of the Internet, have to load all of Watson's information manually, which includes encyclopedias, thesauruses, dictionaries, books, screenplays, and other compendiums of human knowledge.

There will be no audio or video clues in the eventual game, though the questions that require betting--Daily Double and Final Jeopardy--will remain. Watson performs a risk analysis on the categories given for those types of questions, though his precise reasoning means that his wagers are often unusual figures (a human might bet $2,000 instinctively, but Watson's risk assessment might indicate that a bet of $1,986 is more prudent). Watson actually learns in real-time, within the category--if it doesn't immediately understand a category, it will wait until a question or two in that category has been asked, and then use that data to figure out the pattern. Watson also takes the competition into account: If it's losing, it might adjust to answer questions with which it has less confidence than if it was sitting on a large lead.

I spoke to David Ferrucci, the Principal Investigator for Watson's DeepQA Technology at IBM, about the things Watson struggles with. "The things that are most difficult for Watson," he said, "are the things that haven't been written about." Small items that may stick in a person's mind that could lead to the answers of trivia questions are not nearly as accessible to artificial intelligence programs like Watson, even with its massive memory bank.

Certain elements of human language are tricky, too--the stuff that seems like it might be the most difficult (like puns and wordplay) are felt out by "trigger" words in the category name, such as "sounds like." But synonyms are often a bigger problem. In the answer "This liquid cushions the brain from injury," Watson has to determine that "liquid" is in this one case interchangeable with "fluid," and that "cushions" is interchangeable with "surrounds." Humans know what the question is asking instinctively, but Watson has to analyze it from every angle.

In the preview match I saw, which was all too quick, Watson performed surprisingly well. Not just well; it won handily, with $4,400 to Ken Jenning's $3,400 and Brad Rutter's $1,200. None of the contestants, human or machine, actually got a question wrong, but Watson seemed to be fastest at chiming in. Its weakest category was "Children's Book Titles"; Ken Jennings nearly ran the category, and Brad Rutter later quipped that "Neither Watson nor I have kids."

The eventual contest will be a two-day tournament format, in which the competitor with the most amount of money after two days will be crowned the victor. The winner will be awarded $1 million, second place $300,000, and third place $200,000. IBM will donate the entirety of Watson's winnings to charity, while Ken Jennings and Brad Rutter will donate half of their winnings.

Who will emerge victorious with such a big purse on the line? Will flesh and blood greed add caginess to the human competitors, giving Watson an advantage? What anecdote will Watson produce during Trebek's condescending interview segment? (Trebek says he'll "probably try to have a little fun with him.") We'll have to wait until February 14 to find out.

Remember Watson? We're currently at IBM's offices watching the world's best Jeopardy-bot take on Ken Jennings, the winningest human to play the game. Follow @PopSci on Twitter right now for the live blow-by-blow, and stay tuned for a full report later today.

Alex, I'll take "OMG So Excited!!!" for $2,000

Creating an AI that can compete on Jeopardy! is an incredibly difficult task for any programmer. The venerable show poses questions not only as simple shows of trivia knowledge, but also puns, various forms of wordplay, trick questions, riddles, and other complex queries. It takes uniquely flexible and quick thinking to succeed at Jeopardy!, placing Watson on a pedestal with chess-playing robot (and IBM sibling) Deep Blue.

On February 14th, Watson will go head-to-head-to-head on a very special episode of Jeopardy!, playing against trivia legends Ken Jennings and Brad Rutter. (In case you were wondering, Watson's spot at the challenger podium will be taken by an avatar--no word on what strained anecdote Trebek will coax out of him and then casually mock.) Watson has been prepping for the battle by sparring with other Tournament of Champions competitors, though neither IBM nor Jeopardy! has released the trivia-bot's record against human competitors. You can see in the video above that some wordings can trip him up, so nobody knows exactly how capable a competitor he'll be.

The winner will receive $1 million--if Watson wins, IBM will donate the money to charity, and both of the human competitors have pledged to donate half the prize if either wins.

[IBM via AP]

As PopSci reported last year, Watson was designed to decipher Alex Trebek's riddles, tolerate his pronunciation of French words, evaluate its confidence about possible solutions and ring in with an answer -- all within seconds. Now you can beat it at its own game.

IBM scientists started practicing with Watson last winter, and the computer won several matches against frustrated humans, as the New York Times Magazine reports.

Watson evolved out of IBM's DeepQA research on natural-language processing, a means of digitally parsing the information in human communication. The computer is intended to be the world's most advanced "question answering" machine, able to understand a question posed in everyday human language and respond with a precise, factual answer.

Operating at petaflops of parallel processing power, Watson thumbs through its databases, which include scanned textbooks, to come up with the best answers.

Watson is apparently scheduled to make a Jeopardy appearance this fall, but you can make yourself feel smarter (or dumber) today, thanks to the Times' little preview.

[New York Times]