Ruth Given has spent the last few months doing an exhaustive study of the physician ratings business. Ruth is an independent health economist and consultant who has in the past worked for Kaiser, the California Medical Association and Deloitte Consulting. We’re very happy to make her study available on THCB and the Health 2.0 Blog. You can download the full report at the end of this introductory article — Matthew Holt
The past few years have seen an explosion in growth of websites allowing patients to review/rate (usually rant or rave about) their health care providers. Recent mainstream media attention has focused on the rating of physicians, with over 30 such sites now operating. A few sites, including RateMDs and Healthgrades, have been around for a number of years, but several high profile initiatives were recently launched. Last fall, national health plan Anthem announced that it would be partnering with restaurant rater Zagat to allow its enrollees to rate their MDs online. And in April, Angie’s List, whose subscribers rate a wide variety of local service companies, began to include all types of health care providers, including physicians.
Physician reaction to these sites has been generally unenthusiastic; but there is currently very little MDs can do legally to stop patients from posting opinions about them online. While this approach to reporting on MD performance has its shortcomings, there is also a growing recognition of the importance of accounting for patient experience in evaluating quality of care. The federal government, through the Agency for Healthcare Research and Quality (AHRQ) is moving to collect patient experience-related feedback, such as that included in their annual consumer assessment of hospitals reports. An AHRQ/Consumer Assessment of Healthcare Providers and Systems survey tool on patients’ experience with physicians has also been developed and is currently in use in a number of settings.
Given the recent ramp-up in sites and their newly legitimized role, the future for online MD rating seems fairly rosy. But is this really the case?
What are reasonable expectations for the performance of specific sites
and the overall space? Just because we’ve seen a flurry of activity and
funding doesn’t mean that this trend has much staying power. The
previous dot-com boom/bust cycle should make us wary of being taken in
by the newest “new thing,” especially in the frequently over-hyped
realm of health care online. The purpose of this informal analysis is to scope out the
prospects for online MD rating space. I do this by considering three
key questions:
1) What is the value (i.e., benefit, controlling for
broadly defined “costs”) for the consumer/patient/user;
2) How fair are
these sites to the MDs being rated (where fairness and accuracy of
ratings are positively associated with value for the consumer); and
3)
Where will the financial resources come from to support operation of
these sites?
This last question is critical, not only because
value/fairness alone will not ensure website survival if no one is
willing to cover the costs, but because the actual source of underlying
funding (e.g., advertising, other clients, and/or sponsors) may create
an impression of bias, and reduce interest and traffic, ultimately
dooming the site itself. While I have tried to be as thorough as possible in
identifying MD rating websites and exploring the major issues, this is
only a preliminary effort, not the definitive analysis on this space.
In fact, I’m hoping that blog-publication will stimulate lots of
feedback on my assessment of this space from those who know as much if
not more than I do about this topic.
To download the report (PDF) click here
Categories: Uncategorized
Since when is a doctor ratings site a public service when it’s an advertising ploy structured around shaming and blackmailing doctors!
I think you lack the actual questions on conceptual domains. I think one factor which affects the patient satisfaction ratings is due to lack of standardized instrument. Internet can help a lot in our health care systems because through online Info MD online the patient can freely expressed their thoughts about how the Health care member handle them.
Happy Holidays!
How exhaustive of a search was done on rating sites? The study missed 500 city .MD rating web sites, one in every major city in the USA, with over 20,000 ratings in the mid-west alone since August 2007 which were designed to mitigate patient/doctor issues privately.
Thought The Health Care Blog should be the first to make your audience aware that at http://www.Boston.MD or http://www.NYC.MD or http://www.Cincinnati.MD or any of our other 500 .MD city websites in the USA (see all cities at http://www.MyCity.MD), all licensed health care professionals automatically receive and are set up already (knowingly or not over 1 million accounts are up live online) with a free website presence, including patient ratings and free private mitigation tools, for issues/problems, many other free physician desktop tools and free Internet presence and ratings. (No catch!). Read the June 2008 article on one of the Cincinnati.MD doctors that had their entire surgical teams mal-practice fees lowered 3% using .MD survey tools: http://www.healthleadersmedia.com/content/PHY-214175/Turning-Online-Physician-Ratings-into-Lemonade##
Readers also need to know that the city .MD sites are built for the new transparency acts for health care and include ratings and feedback of not just doctors, dentists, chiro’s etc but also hospital and nursing home compare stats, as well as free hospital digital exit survey apps. These apps and programs include but are not limited to the HCAHPS surveys (in http://www.Cincinnati.MD up live now and coming to all hospitals in January 2011) and our just introduced, “Rate Your Health Insurance Company” feature available today in all 500 .MD cities. Try it now in any city from the homepage search box.
Isn’t it time we all had a say regarding our health insurance providers too. After all, turnabout is fair play! We rate and leave feedback for our doctors and hospitals.
Many extremely expensive survey and rating/Coaching companies like Press Ganey and/or Health Grades, etc. will likely hate the new business model but we feel it is for the good of our health care communities to provide all these services, FREE to all of our patients, doctors and hospitals, which have been taking the brunt of the economic health care crunch to date that we’ve been experiencing in U.S. Health Care marketplace. If we help the providers reduce costs, then we believe, the patient will benefit greatly in the long run with better services and much lower costs for themselves as insurance coverage pays less and less each year on the new plans!
Your thoughts?
Thanks,
Joe Benza, Jr.
CEO YourCity.MD LLC
http://www.YourCity.MD
JosephBenza@YourCity.MD
Thanks for your excellent site. I have found a similar site containing excellent information on SEO Services and Online Marketing Strategies at http://bradholton.net
Regards,
Bradley Holton.
Cool list. Definitely some useful links there. I want to inform you that if you wish to know about Baby Bedding and Nursery Room Décor, just visit http://www.bebebedding.com/ Thanks
WoW! I guess there aren’t too many physicians rating this on line rating system. Has anyone stopped for a moment to look at what people, for the most part, complain about. I guess not!
Satisfied patients leave . . . yes, they are satified and not likely to praise or complain. But you can bet the dissatisfied patient will bitch, complain and moan.
Who is likely to be dissatisfied?
Do any of you have any idea what it is like to work in an emergency room, dealing with drug seekers, malingers chronic pain patients and, of course, the truly sick and infirmed all while trying to sort out the sick from the nonsense. Frequently it is amazingly difficult to separate them all. So who do you suppose is more likely to register a complaint when they don’t get what they want. Hmm! Let me think about that. No, there is no thinking. It’s war in the ER.
The docs have to be focused, objective and “on all the time”. They need to forulate differentials and know when to say no and when to say yes based on the presenting signs and symptoms. Yet many of you (the readers) feel an online complaint register fairly represents the “ratings” a physician deserves.
Well, I humbly disagree.
Peer review committees may review physician misconduct for hospital based physicians when complaints are filed. The physician is, at least, afforded some minimal process to defend the complaint.
State medical boards have the ability and power through the Medical Practice Act (a statute enacted by the state) to evaluate more egregious conduct, or any complaint for that matter. The physician, again, is afforded some process to defend the complaint. If the physician is found to have violated some standard of care or is liable for some other misconduct one may view that violation on the state board web sites (after an appropriate review of the complaint to see if there is, in fact, any merit to the complaint.
No tears please.
Ruth,
You did a fantastic job on this very thorough and well-informed report.
We launched Doctor.com in September ’08 and have been astonished at how quickly we were able to grow traffic growth and gain traction with both end users and health professionals. Clearly, there is immense demand for an online solution to the doctor selection process.
As doctor rating sites proliferate it will be interesting to see if anyone tries to develop a universal standard for these ratings, or at least a standardized format in which basic rating data can be shared.
The online real estate space is in the midst of a movement like this, with several of the most popular “Real Estate 2.0” sites working together on listing standards that will be common across all their APIs.
There are pros and cons to the “open data” approach and it takes careful planning and strategic evaluation on behalf of the companies involved to justify sharing even a limited portion of their proprietary information. Whether or not this will happen in the doctor ratings space will probably come down to how significantly sites differentiate from one another. Data is easier to share when the value one delivers is based on how the data is presented and used rather than the contents of the data itself. We have some novel ideas in this area that we’re working on and I’m sure others do as well. If this does come to pass, I could see it being immensely beneficial to patients and very much in keeping with the “Web 2.0” community-driven mindset of many new entrants into the space.
I’m also curious to see if there’s more growth in the peer review space. We have been thinking of some interesting ways to allow doctors to anonymously evaluate their peers (both directly and indirectly). There are some slippery slopes to climb with any peer review initiative but, again, there is also potentially great upside both for the end user and for providers who have worked hard to gain the respect of their peers and the medical communities in which they participate.
While the space is currently buzzing, I believe there is still room for much innovation. At some point we will cross a “tipping point” where the growing trend of patient empowerment will align with increased data availability and a critical mass of physicians who are web-savvy. When this convergence takes place, I think it has the potential to redefine the doctor-patient relationship as well as the field of medical marketing.
I’m glad I came across your report and would be happy to participate in the next iteration of it on behalf of Doctor.com, assuming you do a follow up.
Dr Abdel Meri is the best doctor we’ve been to, me and my husband (Rita J.& Carlos J.). He always gets the diagnosis right, his educational background makes him very knowledgeable. Not only he treats his patients right, he is very patient and respectable even on the busiest days. No matter how busy he is, he takes time with his patients, and helps in anyways he can. His staff are also great and always helpful. When you get in his clinic, you can feel comfort no matter how sick you are, or how much pain you have, because you know you will get treated right.
It is rare nowadays to find a doctor that understands and feels your pain, and take time to listen to your problems, I have even seen some doctors spend at the most FIVE minutes for a first office visit !!!.
I truely and strongly recommend Dr Abdel Meri to every patient, he has the knowledge and the skills to not only give you a diagnosis and a treatment, but also to comfort you. I am sure that when you walk into his clinic, all his staff will treat you with respect. So just call his clinic and you will be happy with your decision, the way me and my husband are.
Wait a minute! The world is flat, right? At least this is buzz seemingly with traction and a home in the blog and micro blogospheres…We’ve all heard of UGC (user generated content) as “the next, new thing”.
Yet there is no better example of hierarchical overkill than in the health care industry. While most of the quintessential bureaucracies are hosted by institutional providers, hospitals, healthplans, integrated delivery systems,etc., make no mistake mainstream medicine has done their level best in architecting a “special boy/girl” syndrome around the coveted “MD-iety” status.
So why not level the playing field? After all isn’t the professional invitation….”first, do no harm”?
How presumptive that the doctor knows best or all for that matter. Until their is a leveling with eyeball to eyeball HUMAN connection (not one a “one up, one down” exchange relationship), docs will continue to separate themselves from their patients and watch in horror as powerless patients “get even” by voicing their opinions on MD rating sites.
“The report mentioned the biggest online rating site only has approximately 10,000 ratings.”
I wonder if John Grohol even read the report. I further wonder if the good doctor is having his judgement colored due to being rated negatively on at least one of the sites. But maybe I’m just cynical.
Ruth,
HealthGrades has a bit of data on your question that I can share.
A recent survey of our visitors finds that 31% are searching for information on their current doctor. The rest are searching for information on a new doctor — 47% know that doctor’s name, and 22% are researching new doctors without knowing a specific name.
That 31% is interesting, with respect to Jane’s opinion; a healthy portion seem to simply be checking up on their current doctor.
I’m glad to see that the major Internet survey research initiatives (esp. Pew) are planning more questions about patients’ interests in MD info online. Based on my recent research experience, I have a couple of suggestions.
1st – it would be great to know more specifically under what circumstances patients find this info most helpful. When I talked to Jane Sarasohn-Kahn a couple of months ago, we had rather different opinions. She thought it would be mostly when there was a need for super-specialty care for a high cost, high risk condition – but pretty much a one-off situation.
While I agreed re this need, I also thought that there was a growing hunger for an online version of “over-the-back-fence” source of info on primary care providers one sees on a regular basis – esp. when people move into a new community or as aging baby boomers realize chronic concerns can no longer be ignored. Knowing the breakdown of prospective users should allow sites to tailor info provided to better meet their needs.
2nd, and I think even more important to know than why people SEARCH for this info is why people decide to POST (or not) opinions about their MDs. A large (and ideally representive) database of opinions about MDs is what drives consumer value for these sites. As a result users should be considered an essential component of the production process for this business – understanding what motivates them is critical for success.
As I pointed out in the report, all of the official/objective info on MDs is available from other sources, usually for free and typically in a more reliable form (at least in California). Better understanding the motivation for and dynamics of MD rating (user generated content) could result in public and website policies that boost participation, increasing the accuracy/fairness of ratings and the value the sites provide to all.
Susannah,
Yes, HealthGrades has traditionally been of most use to patients facing a medical condition who are in need of finding the right doctor, hospital nursing home for themselves or a loved one. We do recognize that not everyone’s online, however, which is one reason why we published a book this year with Sterling: http://search.barnesandnoble.com/HealthGrades-Guide-to-Americas-Hospitals-and-Doctors/From-Healthgrades-the-Leader-in-Healthcare-Ratings-with-Samantha-L-Collier-MD-MBA-Healthgrades-Chie/e/9781435104266/?itm=1.
We’ve also recently puchased two other sites, wrongdiagnosis.com and cureresearch.com, that have broader health content, vaulting HealthGrades into top-ten health property status as we attract a wider user base.
Really looking forward to the results of your current research.
Scott,
I thought your strongest point in Boston was when you pointed out that people facing serious health issues are focused on finding the right doctor. I’ve seen the same thing in my own research comparing the worried well and those living with chronic conditions.
But here’s what I just wrote over on e-patients.net where we are discussing the same topic:
Another issue in the development of the doctor-rating sites is that many people don’t have a wide range of choices when it comes to their providers AND once one is chosen, most people won’t need the service again for years. It’s an example of the point Stead Burwell made to me when we met at Connected Health in Boston: health is a topic that many people engage with (80% of internet users) but few on a daily basis (7% on a typical day).
The Pew Internet Project’s surveys have found (in 2004 and 2006) that 29% of internet users have looked online for info about a doctor or hospital. Our upcoming survey will separate those two and get a clean read on how many internet users are looking up doctors. The measure will be more of a “have you ever” question, though, not a frequency question.
So you are targeting the slice of the population that is both online and facing a serious diagnosis? That is, unfortunately, a growing sector.
We are impressed with your keen perception of the rating sites and the report was truly exceptional. As part of the Vitals team, we are glad that our cumulative efforts have shown through. We continually strive to make it easier for users to find a physician that is right for them based both on empirical qualifications (i.e. quality of education, board certification, etc.) as well as subjective qualifications (i.e. ratings, reviews, etc.).
As an aside, at the time you wrote the article it seems that you couldn’t find Dr. Monique Marie Baer on the Vitals.com search page. You can now find the profile for Dr. Monique Marie Baer by searching on her many name variations including common misspellings and nicknames. We are updating the Vitals.com site regularly to provide the most accurate, comprehensive and user friendly site for our users.
Thank you,
Irving Weiss
Thanks to Ruth for the thorough review of the doctor-ratings organizations.
As she mentions in her post, rapid aggregation of surveys is key, and HealthGrades is working on extending its lead in this area.
Right now, through our partnership with Google and other initiatives, HealthGrades receives more than 6 million visitors a month, and it’s one of the Web’s top ten ad-supported health sites. That’s crucial for amassing patient-satisfaction surveys, of which we have north of 600,000 today. We expect that the just-announced partnership with Yahoo! Health, under which HealthGrades will be the exclusive provider of doctor and hosptial information, will send those numbers higher next year. We are also actively licensing our content to other organizations (you’ll find it on aarp.org, and on the employee portals of Microsoft and the Gap, for example) to get this information in even more people’s hands.
On a separate note, I would caution people against citing Harris and other surveys that show low utilization of doctor ratings. It’s my understanding that they surveyed a general population, the majority of which are lucky enough to have absolutely no need of, or interest in, doctor ratings. (That’s a good thing.) What’s important is the subset of people who actually have a medical need; it’s those people who are using the ratings sites — and in large numbers, according to our data.
I agree with Dr. Grohol that there are not nearly enough MD ratings out there to make these sites very valuable for patients thus far. As I mention at the end of the analysis (invoking Clay Shirky) the “plausible promise” will only be fully demonstrated if the sites continue to “grow” their user generated content.
One correction – the site with the largest number of ratings (RateMDs) has over a half a million. The 10,000 ratings mentioned were those acquired from a smaller rival site last fall.
I don’t indicate their total number of ratings in the text, but for more info on RateMDs experience (by state/province in the US and Canada) see Table 10. Via RateMDS, Canadians have rated nearly 60% of the MDs in their country, while in the US the percentage is just 12%. Still room for much improvement in both countries, but this difference really got my attention and made me wonder why the Canadians are so far ahead of us. I’m hoping blog readers have some suggestions beyond the few ideas I could come up with.
Re the problem of many separate sites that limit rapid aggregation (optimal for value) – I also agree. The presence of network externalities makes this a natural monopoly. That’s why I was puzzled that instead of cooperating, the various Blues plans seemed to be going their own separate ways on MD ratings (at least Anthem/Zagat, Regence and BCBS of Minn). Given the need to get big fast, doesn’t it make more sense to cede this task the national BCBS organization?
That’s another reason to think that a single, highly trusted organization (like Consumers Union) might be the best sponsor for MD rating activity.
Steve, that’s a great start, resulting in a fairly decent 0.008 to 0.01 sampling rate. You’d only need 7,000,000 ratings at that level to have ratings on more than half of the physicians in the U.S.
The report mentioned the biggest online rating site only has approximately 10,000 ratings. And this is after these sites have been online, in some cases, for years. So the biggest site is 0.15% of the way there!
And since each site has no data sharing with any other site, your site (and every site like it) is setting up a nice balkanization resulting in no site getting sufficient numbers of ratings to become a clear ratings leader.
Read my critique at e-Patients.net for more issues with these services. Apparently few statisticians were employed when thinking through these significant validity issues.
Suzannah Fox wrote:
Yes Suzannah – only 7 note-words though
😉
This is a terrific analysis of the doctor rating websites that are available. At DrScore, we believe these sites will grow strong because they benefit both doctors and patients. Making doctors’ scores public will help people see the quality of care being given by American physicians, who– despite appearances sometimes– are dedicated to their patients.
We looked at the doctors with 20 or more ratings on the DrScore.com site and the median score was well over 9 (on a 0-10 scale where 10 was the highest score). We’ve seen steady growth in the number of doctors who actively send patients to DrScore for feedback. Doctors have little to worry about from online ratings.
One thing I was hoping to see in the report was a detailed mapping of the actual questions to conceptual domains. The one and overriding factor inhibiting patient satisfaction ratings is the lack of a standardized instrument for actual patient satisfaction ratings.
This problem has largely been addressed with hospitals through the creation of the HCAHPS instrument for hospitals and a mandate by CMS that hospitals must comply/have their results publically reported on the CMS website. AHRQ has also created a CAHPS survey instrument for medical groups that has been well validated and vetted but I would have to check on the status of a CAHPS instrument for individual providers.
Other question that really fundamentally matters is patient satisfaction linked to improvements in clinical quality? The literature here is really scant but this still hasn’t stopped some early P4P programs for including a patient satisfaction component to their P4P payment algorithm for medical groups and hospitals to a lesser degree.
One area that I have been particularly interested in is how you take the qualitative reviews by individual reviewers and summarize this information into a useful form for consumers. Guess most sites though have to walk before they run and actually set a minimum statistically signifcant sample size for a number of physicians though.
Quick correction: The medical section of Angie’s List is in beta! Obviously the rest is not.
John Grohol just posted a review of Ruth Given’s report on e-patients.net:
How Good Are Doctor Rating Sites?
http://e-patients.net/archives/2008/11/how-good-are-doctor-rating-sites.html
This was also the topic of a panel at the Connected Health Symposium in Boston: “Giving Out Grades: Reports from the Field on Online Provider Ratings” with Paige Amidon of Consumers Union, Angie Hicks of Angie’s List, and Scott Shapiro of HealthGrades.
Did anyone else attend that session? I wrote down notes that wouldn’t surprise anyone: Amidon called for more transparency, disclosure, and standardization; Shapiro said to bet on consumers to use ratings despite a Kaiser study AND a Harris study showing low awareness/interest; Hicks emphasized that Angie’s List is a supplement to other sources and it is in beta (meaning: please help us get it right).
Wow! A rather level-headed perspective is published here?
Color me impressed.