One of the computer applications that has received the most attention in healthcare is Watson, the IBM system that achieved fame by beating humans at the television game show, Jeopardy!. Sometimes it seems there is such hype around Watson that people do not realize what the system actually does. Watson is a type of computer application known as a “question-answering system.” It works similarly to a search engine, but instead of retrieving “documents” (e.g., articles, Web pages, images, etc.), it outputs “answers” (or at least short snippets of text that are likely to contain answers to questions posed to it).
As one who has done research in information retrieval (IR, also sometimes called “search”) for over two decades, I am interested in how Watson works and how well it performs on the tasks for which it is used. As someone also interested in IR applied to health and biomedicine, I am even more curious about its healthcare applications. Since winning at Jeopardy!, Watson has “graduated medical school” and “started its medical career”. The latter reference touts Watson as an alternative to the “meaningful use” program providing incentives for electronic health record (EHR) adoption, but I see Watson as a very different application, and one potentially benefitting from the growing quantity of clinical data, especially the standards-based data we will hopefully see in Stage 2 of the program. (I also have skepticism for some of these proposed uses of Watson, such as its “crunching” through EHR data to “learn” medicine. Those advocating Watson performing this task need to understand the limits to observational studies in medicine.)
One concern I have had about Watson is that the publicity around it has been mostly news articles and press releases. As an evidence-based informatician, I would like to see more scientific analysis, i.e., what does Watson do to improve healthcare and how successful is it at doing so? I was therefore pleased to come across a journal article evaluating Watson . In this first evaluation in the medical domain, Watson was trained using several resources from internal medicine, such as ACP Medicine, PIER, Merck Manual, and MKSAP. Watson was applied, and further trained with 5000 questions, in Doctor’s Dilemma, a competition somewhat like Jeopardy! that is run by American College of Physicians and in which medical trainees participate each year. A sample question from the paper is, Familial adenomatous polyposis is caused by mutations of this gene, with the answer being, APC Gene. (Googling the text of the question gives the correct answer at the top of its ranking to this and the two other sample questions provided in the paper).
Watson was evaluated on an additional 188 unseen questions . The primary outcome measure was recall (number of correct answers) at 10 results shown, and performance varied from 0.49 for the baseline system to 0.77 for the fully adapted and trained system. In other words, looking at the top ten answers for these 188 questions, 77% of those Watson provided were correct.
Continue reading “Chapter Nine: In Which Dr. Watson Discovers Med School Is Slightly Tougher Than He Had Been Led to Believe”
Filed Under: Tech, THCB
Tagged: EHR, IBM, Informatics, Meaningful Use, Watson, William Hersh
Jun 15, 2013
This past week, Google had its annual developers conference, Google I/O. One of the more provocative talks, called “The End of Search as We Know It,” was by Amit Singhal, who is in charge of search for Google.
The vision, as described by Amit, is that instead of typing words into a box on a website or mobile app, we will have conversations with Google, enabling a much more personalized, refined experience. The holy grail, of course, is that Google analytics become both predictive and prescriptive, serving you content that is just right for you and anticipates your needs.
It seems there is a race on now to achieve this vision. One could argue that Amazon, Apple, Facebook, Pandora and others are all in the same mode. Best I can tell, the promise these companies are floating to advertisers is that their ads will be served up to that focused slice of the population that will find their product relevant in the moment.
If you apply this thinking to healthcare, several controversies/topics come to the fore.
Is Google competing with IBM’s Watson? Undoubtedly yes. On the other hand, I’m guessing Google is disenchanted with the consumer health space after the demise of its personal health record (PHR). And IBM seems to be focused on clinician decision support. So early in the game, with respect to healthcare anyway, maybe there is not much competition. The path for clinician decision support is clear and the market obvious, whereas the path and market for consumer health decision support are blurry.
Continue reading “Is the End of Search the Beginning of Personalized Prevention?”
Filed Under: Tech
Tagged: FutureMed, Google, IBM, Joseph Kvedar, Personalized Medicine, PHR, prevention, Watson
May 23, 2013
Technology is transforming health care in many ways. CEOs of health care businesses think the biggest transformation in the next few years will come from making patients, doctors and health-care workers more communicative and collaborative.
They foresee patients with the same rare diseases coming together in online social networks where they can discuss their symptoms. They see overweight consumers building mutual support networks to share diets and praise exercise. They anticipate that knowledge will be shared so that nurses, pharmacists and social workers can often perform tasks that today are handed to doctors by default.
Every year, IBM surveys hundreds of CEOs from around the globe about a variety of issues. Among 1,700 CEOs surveyed this year there were 58 who head hospitals, medical practice groups and insurers.
The CEO perspective is interesting, because most outsiders don’t think of collaboration as being a key outcome of medical technology. Most of us think of laser-guided surgical instruments or designer drugs or computerized analytics that spot hitherto unnoticed disease-causation chains.
The CEOs overall see technology as a way to open up their organizations to create value through collaboration. Making the organization more transparent makes it easier to share cultural values and goals. And that makes employees more receptive to tough changes, because they understand what’s behind the plan.
Continue reading “Healthcare CEOs Weigh In On Technology and the Growing Importance of Social Collaboration”
Filed Under: Health 2.0, THCB, The Business of Health Care
Tagged: CEOs, health social networks, IBM, Innovation, Internal Communications, Mohamad Naraghi
Oct 5, 2012
In the next 10 years, data and the ability to analyze the data will do for the doctor’s mind what x-ray and medical imaging have done for their vision. How? By turning data into actionable information.
For instance, take Watson, IBM’s intelligent supercomputer. Watson can analyze the meaning and context of human language, and quickly process vast amounts of information. With this information, it can suggest options targeted to a patient’s circumstances. This is an example of technology that can help physicians and nurses identify the most effective courses of treatment for their patients. And fast: in less than 3 seconds Watson can sift through the equivalent of about 200 million pages, evaluate the information, and provide precise responses. With medical information doubling every 5 years, advanced health analytic systems technologies can help improve patient care through the delivery of up- to-date, evidence-based health care.
Continue reading “Now you have healthcare data. So where does it go?”
Filed Under: THCB
Tagged: ACOs, Data, IBM, PCMH, primary care
Jan 27, 2012
Little over a month ago, IBM and WellPoint announced an agreement wherein WellPoint will deploy IBM’s latest and greatest super computer and artificial intelligence mega-mind Watson. Watson’s claim to fame was its ability to beat the human Jeopardy champions much like Big Blue beat reigning chess champion Garry Kasparov in 1997. Since that Jeopardy match, IBM has been quite vocal about its desire to apply Watson in the medical arena, we’ve been buried in press releases and briefings, but the WellPoint announcement is the first one of any real consequence. Having interviewed both IBM and WellPoint, following is our review and assessment.
Watson is a relatively new form of artificial intelligence, based to some extent on neural networks. What is unique about Watson is that it has been developed (trained) to understand the nuances of language. It is a question & answer system that uses among other techniques, natural language processing, to extract meaning out of unstructured data. In developing Watson for the Jeopardy challenge, one of the key design parameters was for Watson to answer a question in under three seconds – plenty fast enough in a diagnosis/treatment decision scenario. This is a key reason why Watson may have enormous utility in the healthcare sector where so much data is unstructured, the pace of change is so high and the ability to chose the optimum treatment patient plan for a given diagnosis is less than ideal today.
Continue reading “Dr. Watson I Presume”
Filed Under: Uncategorized
Tagged: HIT, IBM, John Moore, Watson, Wellpoint
Oct 18, 2011
Health care is in the process of getting itself computerized. Fashionably late to the party, health care is making a big entrance into the information age, because health care is well positioned to become a big player in the ongoing Big Data game. In case you haven’t noticed computerized health care, which used to be the realm of obscure and mostly small companies, is now attracting interest from household names such as IBM, Google, AT&T, Verizon and Microsoft, just to name a few. The amount and quality of Big Data that health care can bring to the table is tremendous and it complements the business activities of many large technology players. We all know about paper charts currently being transformed via electronic medical records to computerized data, but what exactly is Big Data? Is it lots and lots of data? Yes, but that’s not all it is. Continue reading “The Rise of Big Data”
Filed Under: Health 2.0
Tagged: AT&T, Big Data, Google, IBM, Microsoft, Social Media, Verizon
Oct 10, 2011
IBM’s Jeopardy-champion computer, Watson, has huge potential for helping physicians and other clinicians work with patients.
The leap from TV game show to physicians’ offices will probably take at least two years. But Watson’s understanding of natural language, vast storehouse of information and ability to keep up with rapidly changing medical research could significantly improve medical care.
The medical faculty at Columbia University and University of Maryland are helping program a Watson-type computer to assist clinicians.
A few years from now, consulting Watson could become a routine part of a clinician’s practice. Caregivers have traditionally resisted computerized assistance in diagnosis and treatment because the technology has been awkward to use and questionnaire-based systems have been too rigid. But Watson can “understand” descriptions of a patient’s symptoms in natural language, and it can even scan years of medical records and doctors’ notes to determine what diagnostic and therapeutic options it might suggest. Doctors can ask it questions using the same terms they would use in an e-mail to a colleague. Continue reading “From Jeopardy! To Your Physician’s Black Bag: Could a Supercomputer Really Assist With Health Care?”
Filed Under: Uncategorized
Tagged: Dan Pelino, IBM, Medicine, Physicians, Watson
Mar 14, 2011
Game Show Watson wants to be a doctor. Well, almost.
Fresh off a commanding victory on Jeopardy, IBM will try to demonstrate that the combination of advanced natural language processing and sophisticated algorithmic decision-making capabilities involved in its extraordinary Watson computer can help humankind, not merely humiliate human competitors.
As I wrote on a previous blog, IBM began eying the medical marketplace more than 45 years ago. IBM CEO Thomas J. Watson, Jr. – son of the IBM CEO for whom this computer was named – put it this way in 1965: “The widespread use [of computers]…in hospitals and physicians’ offices will instantaneously give a doctor or a nurse a patient’s entire medical history, eliminating both guesswork and bad recollection, and sometimes making a difference between life and death.”
Now, IBM is ready to turn that vision into reality. At heart, Watson is the world’s most sophisticated question-answering machine. The company is collaborating with Columbia University and the University of Maryland to create a physician’s assistant service that will allow doctors to query a cybernetic assistant. IBM will also work with Nuance Communications, Inc. to add voice recognition to the physician’s assistant, “possibly making the service available in as little as 18 months.” For Nuance, it could be a major business line, and promises to carry over in the not too distant future to the mobile phone market, such as Apple’s iPhone, where Nuance is a major presence. Continue reading “Watson: A Computer So Smart, It Can Say, “Yes, Doctor””
Filed Under: THCB
Tagged: IBM, Medicine, Physicians, Watson
Feb 17, 2011
If you want to see the future of health information technology, take a look at the dueling visions of two Thomas Watsons that are on display this month in a game show and a trade show. The juxtaposition unintentionally demonstrates what doctors and patients will be doing together and also what they can do separately.
What I’ll call Game Show Watson is a computer named for IBM founder Thomas J. Watson, Sr. This Watson is appearing on the TV show Jeopardy to play a highly publicized set of matches against two human champions from Feb. 14-16. Although viewers will actually see a black computer screen with a revolving blue globe, Game Show Watson itself, in the tradition of “Big Iron” mainframes, consists of ten refrigerator-sized servers located offstage.
In contrast, the Watson at the trade show is not one computer, but thousands of them, all contained inside the mobile devices that are descendants of the telephone first demonstrated by Alexander Graham Bell and his assistant, Thomas A. Watson. (That Watson was also an inventor is a topic for another time.) The Telephone Watsons, on display for the tens of thousands of attendees at HIMSS11 from Feb. 20-24, are giving rise to a new field known as “mobile health.” Continue reading “Health IT Future: A Tale of Three Watsons”
Filed Under: OP-ED
Tagged: IBM, Mobile health, Watson
Feb 15, 2011