Are Mystery Shoppers Such a Bad Idea for Health Care Quality Improvement?

The decision by the Obama administration to employ “mystery shoppers” to pose as patients to see how difficult it is to get an appointment with a physician has sparked criticism from physicians. However, access to primary care physicians is a very real public policy issue that needs to be understood if we are to successfully care for the more than 30 million Americans who receive coverage under the Affordable Care Act.Is the use of “mystery shoppers” a bad idea?

Dr. Raymond Scalettar certainly thinks it is a bad idea. “I don’t like the idea of the government snooping. It’s a pernicious practice – Big Brother tactics, which should be opposed.”

Dr. George Petruncio says, “This is not the way to build trust in government. Why should I trust someone who does not correctly identify himself.”

Westby Fisher, MD writes in his blog: “When information gathering trumps patient care – particularly fictitious care – we’ve got a problem. Is this a new quality standard we can expect from our new government health care initiative?  Just like scam-artists that phish for unsuspecting people’s financial information online, governmental appointment phishing should not be tolerated in any way, shape, or form. It is fraud – plain and simple.”

Several physicians on twitter retweeted Dr. Fisher’s blog post and indicated they agreed with his analysis.


A mystery shopper is a quality improvement tool that has been used in the retail and hotel industry for decades. In 2004 the health care industry accounted for 2% of mystery shopper revenue, but the use in health care is increasing.

In health care, mystery shoppers have been used to provide feedback on every step in the patient experience: making an appointment, environment of the waiting room, encounter with nurses and office staff, and visit with the physician. There has been an increased interest in mystery shoppers by health care providers since CMS started posting patient satisfaction information on hospitals as part of the Website Hospital Compare.

The use of mystery shoppers has been reported to lead to better patient flow and improved wait times, extension of office hours, improved telephone etiquette, better physician communication with patients, and more time with patients answering questions about surgery.

The AMA Council on Ethical and Judicial Affairs studied the mystery shopper question and concluded that a sound program would include:

1) The places that unannounced visits will affect should know about the program.

2) The information should be used for improvement and not punitive actions.

3) Mystery shoppers should not be the sole source of data for evaluating clinical performance

4) The program should not adversely affect access to medical care by legitimate patients.

I disagree with my colleagues that a properly planned and implemented mystery shopper program is a bad idea for trying to improve health care. For far too long, we in medicine have been too arrogant to learn lessons from other industries that improve quality. I think we need all the help we can get to take better care of patients.

Kent Bottles, MD, is past-Vice President and Chief Medical Officer of Iowa Health System (a $2 billionhealth care organization with 23 hospitals). He was responsible for the day-to-day operations of a large education and research organization in Michigan prior to his work with in Iowa with IHS. Kent blogs at Kent Bottles Private Views.

35 replies »

  1. Now it is free to anyone who wishes to use it and it has become one of the most popular free web based e-mail services on the internet.
    If this happens with you almost daily then the suggestion would be
    to carry out Gmail backup to PST procedure. I typically like to enter @domainname I figure I want
    to have the same label on all emails from that domain.

  2. Thank you for some other informative website. Where else may just I get that kind of information written in such an ideal approach? I have a undertaking that I am simply now operating on, and I have been at the glance out for such information.

  3. I agree with most of what you’re saying. I just posted something similar to what you’re talking about on my blog.

    Here’s part of what I said about the types of automated trading robots you would like to have.
    There are basically 2 types of trading robots. A semi automated and a fully automated. If you want some controls to your trading, you can go for semi automated robot. This robot will create trading signals or what we call buy sell signals and notify the trader about it. The other type is fully automated trading.
    For those who do not have time to or just do not want to hands on and trade the market. This is a very good choice. Since the automated robot can help you to place trades by itself .

  4. Accountability is un-American; particularly for self.

  5. The AMA is also totally in the pockets of the specialists, and exists to shaft primary care.

  6. ROFLMAO! This whole mystery-shopper thing is a sad indication of just how out-of-touch and clueless these Beltway ppl are.

  7. Many doctors in private practice now consider themselves businessmen as well as professionals, using marketing strategies to increase income, focusing their practices on procedures with the greatest monetary “return,” and the like.

    Since they are now businesses as well as medical practices, they should accept that they will be judged partly by business “metrics,” and that means they should submit to mystery “shoppers.” They can’t have it both ways.

  8. As noted above by the author of the post, this disgusting plan has been withdrawn, but you think it was because of common sense, or, they were exposed for the fraud the government is:


    As I have said over and over at this and other sites, what the hell was the AMA thinking when they supported this nightmare legislation?! As my colleague above MD-H said so well, flaming idiots is an understatement!

    The usual diehard apologists and defenders will paint those like me who argue against government intrusion as just being selfish and greedy, but, come on, what healthy persons who need to make an income can have someone else look them in the face and tell them to take a loss for their job and then this “advisor” expect to be thanked for the suggestion? Only a politician and their ilk supporters would stoop to this level of insult!!!

    This isn’t about people supporting this legislation as being simply insincere, they are the poster children for hypocrisy gone mad and obnoxious examples of “do as I say, not as I do”.

    Again, will not let the lie go unchallenged!

  9. Give’em credit for recognizing stupid when it is pointed out to them and deciding to walk away. (If only it were so easy in Afghanistan or Iraq.) Really stupid would have been to continue. One observation is that normally intelligent people captured and held by the Washington beltway for too long will almost always have an acute attack of bad judgment. It must be in the water (after all Washington was originally a swamp!) Acknowledging this is a step in recovery.

  10. What must be remembered is that in Medicaid there are people whose JOB it is to recruit physicians into the program. This is the same whether we are talking fee for service (FFS) or HMO. In the latter, there are network adequecy requirements in the HMO contracts. There are also people whose JOB it is to investigate and resolve client complaints that they cannot get an appointment. Again, this is the case whether it is FFS or HMO. There are also oodles of adminstrative data available to discern if patients are not getting needed doctor visits. Why not use what is already readily available as resources to looki into the matter?

  11. “We really do have a hard time holding anybody accountable in American health care”

    If this survey had proceeded, who would be “held accountable” for what?

  12. “If there is a problem, it just indicates that there is a mismatch of reimbursement between medicaid and other payors.”

    In general you are correct, but it can also indicate a lot of other things. It can indicate that the provider (and we are talking more broadly than just docs) finds the hassles of Medicaid to be too cumbersome. Maybe they dislike the prior auth procedures, feel too many claims are rejected, that payments are late, or the clients are hard to deal with due to no-shows, non-compliance, language or medical literacy issues, a prevelance of MH/SA (mental health/substance abuse) issues above the rest of their panel, that clients lose coverage too often, etc.

    Patient surveys can indeed be flawed if you measure subjective (even nebulous) terms like satisfaction. However. asking a new Medicaid patient if they were able to get PCP after joining the program isn’t rocket science. it is an objective question, with a binary answer. Ditto for asking the ER frequent flyer with asthma why they show no claims for a PCP visit. Target the queries to get right to the problems.

    You call for good basic health care is one of merit, by the way.

  13. The root problem is that there is dishonesty at work. How does one build trust with providers when people on your employ will CALL THEM UP AND LIE TO THEM.

    Next, I simply reject the contention that ASKING THE CLIENTS if they are having problems scheduling appointments or asking doctors if they have panel openings costs as much as hiring a survey firm. Seriously, the Medicaid or Medicare agency is already communicating with both parties. If it through an MCO model (Medicaid Managed Care), then the MCO is contractually obligated to provide an adequate network. If clients can’t get access, they are breaching their contracts. Also, if it is a Medicaid agency, they are taking complaints if a client can’t get access. Simply track the complaints and look for patterns with better data collection at the time of the call. Existing data and existing client and vendor relationships can – and should – be what is used to gather usable data on accessgaps. Sorry, but this is still a complete waste of cash.

  14. What do we want to achieve? Verify access for medicaid patients? If there is a problem, it just indicates that there is a mismatch of reimbursement between medicaid and other payors.

    Clinical quality is very hard to measure. There are some surrogate parameters that medicare is already measuring (beta blockers for MI, time to ABx), more in depth assessment could probably done via independent peer review only.

    Patient surveys are heavily biased by convenience factors such as parking, pleasantness of facility etc, and many patients are easily seduced by perceived “cutting edge” nonsense (as are many physicians). Even communication assessments can be flawed when patient’s don’t like the provider’s message.

    I practice in a competitive metropolitan area (and practiced in an even more competitive one before). Patient satisfaction centeredness brings you quick access (too quick access drives up costs), very polite, pleasant docs (a good thing, but not when docs feel they have to take patient requests for testing, without asking because that could make ’em unhappy), shiny facilities with marble, valet parking and birthing suites, and stupid direct to consumer advertising for the latest technology … it’s good for the wealthy if they want pay for it. The bottom half needs just reasonable, affordable healthcare that is not doing diagnostic& therapeutic overkill.

  15. Patient surveys have their own problems, and if you do them correctly, they cost just as much. Aaron Carroll links to a new article from the Archives of Internal Medicine which claims much higher acceptance of new Medicare patients than is commonly thought. Lower for private patients than one would expect. We cannot make good decisions w/o good data. How else do we get it? The study, as I have seen it described, should resolve a number of problems with methodology issues.



  16. “I fail to see what people are worried about.”

    For one thing, we are borrowing 40 cents on the dollar right now – and this was a wholly uneeded expense. Medicare and Medicaid could get this same data from multiple existing sources, as was already noted above. Just SURVEY THE PATIENTS for gosh sakes! There would be no need to have people be dishonest with doctors and their staff.

  17. Sometimes what everyone knows is not true. If true, it needs to be quantified. Why should we just guess? I fail to see what people are worried about.


  18. “A mystery shopper is a quality improvement tool that has been used in the retail and hotel industry for decades.”

    And, everyone knows that “shopping” for health care is no different than shopping for shoes or flat panel TVs. Just ask John Mackey.

  19. It was an ARRA earmark expenditure from the U.S. Department For The In-Depth Study of Blinding Glimpses of the Obvious.

  20. I have never joined the AMA and now I know why. They are a bunch of flaming idiots who just want to be liked by the government.

    There always have been mystery shoppers. They will only learn what is already known: Mcare and Mcaid and Tricare are not widely desired by primary care doctors. That is not news. Many practices will flatly tell callers they are not taking any new Mcare patients. That is not illegal. Mystery shoppers are unnecessary for that purpose.

    So what is their real purpose?

    J. Edgar Hoover would be proud.

  21. “A mystery shopper is a quality improvement tool that has been used in the retail and hotel industry for decades.”

    But they’re employed or under contract to the retail outlet or the hotel, not the government. Big difference. I think there would be big legal problems with either CMS or an insurer sending mystery shoppers into health care facilities under false pretences.

  22. A more reliable feedback mechanism is surveying the patients themselves. Then interview others with insights into access, such as your disease and case management services vendors and/or your managed care organizations (Medicaid/CHIP uses a lot of them). The latter have network adequacy provisions in their contracts, so they should be reporting access-related stats on this metric anyway — or you should be asking for them. Also, track closely any complaints received regarding an inability to get appointments. Then follow up to see if they were fully resolved. You need to know if REAL patients with REAL Medicare and Medicaid coverage can’t get an appointment. Another item to examine is whether the rates vary depending on whether it is a FFS or HMO model (Medicaid may use both in the same state).

    You can also gain insights from looking into your ER frequent flyer populations, protentially preventable hospitalizations, and potentially preventable readmissions (may flag a lack of post-discharge appointment opportunities). None of these activities requires hiring a new vendor or any dishonesty with the doctor or their staff. All can also typically be done with existing administrative (claims and encounters) data and current vendor relationships.

    What would be most useful would be insights into what has actually worked to increase the availability of appointments (both primary care and specialty). Was it reimbursement increases? If so, how much? Was it personal outreach to the physician to recruit? Was the option to accept a limited number of new Medicare or Medicaid cases explored? Were physicians who didn’t take Medicaid or Medicare asked why? Did they have a problem getting paid? Did they think it was too much hassle (too many prior auth requests? too many rejected claims? Etc.?) Did they get a lot of client no-shows? Was getting any sort of compliance with the treatment plan by the patient overly difficult. Were there language or medical literacy issues?

    Identifying that a provider doesn’t take Medicare or Medicaid or that a patient can’t get a timely appointment doesn’t actually solve anything, in and of itself. You have to get to the “why” part of it all. Any possible solutions will flow from there.

    If anyone does have evidence regarding strategies that worked to improve access, I would welcome a THCB article posted on this topic.

  23. Sillly, silly study.

    We already know we don’t have enough primary care docs. We already know many docs don’t see Medicare patients. We know even more don’t see Medicaid patients. What new info is this little study going to give us?

    Next up: the government hires mystery naturalists to determine if bears defecate in the woods.

  24. I think the main goal for mystery shopper is to improve the quality for the patients, which doesn’t seem like a bad thing and I agree with it.

  25. Every question in the form of “Is X a good idea?” has two types of answers: one for if the government does it, all the others for when the agent is not the government. Many, many actions are sensible except when the government does them.

    At the minimum, sophisticated observers build into their analysis this metaphysical truth: any entity which is, by definition, insulated from normal feedback loops is presumed to be corrupt.

    Mystery shoppers from the government will become the gestapo, no matter their larval form.

  26. What’s missing from the conversation of using mystery shoppers to evaluate access is identifying the question this process is designed to answer. It can only evaluate access for new patients to a primary care practice, which is likely to differ from access for established patients.

  27. A recent study published in the New England Journal of Medicine by Bisgaier and Rhodes, “Auditing Access to Specialty Care for Children with Public Insurance,” indicates that children covered by Medicaid are less likely to get an appointment with a pediatric specialists than a child whose parents have private insurance. It also showed that getting an appointment with a pediatric specialist wasn’t easy either.

    I suspect that a secret shopper study for adults will show similar results. The less attractive the third party coverage the harder it is to get an appointment. The ugly truth is that public insurance coverage is often just a license for a patient to hunt for care. So once we’ve proved this, what are we going to do about it? Increase physician compensation? I don’t think so.

    The number of primary care physicians is in the short run fixed, since it takes 8 years to produce one. It’s only prudent business practice to restrict the number of patients or visits and to limit those visit to patients who compensate providers the highest amount or with least restrictions/ hassle factors. There are taxes and salaries to be paid not to mention the payments on the new EHR software….. Physicians see more than 25 or 30 patients a day burn out.

    Develop a new business model of care — the medical home? Without enough practitioners we might as well call it a medical shack.

    A new business model depends on the incentives of how we pay for care. Until a new payment model is devised that is fair to physicians and patients, barriers to access will remain. Legal actions against physicians who don’t accept new patients is counterproductive and will drive more and more physicians to turn to concierge practices where only an American Express card is accepted. Is that the model we want?

    Spending two years to prove what we all know is going on doesn’t resolve the problem of too few primary care providers (or pediatric specialists). If we value something we count it and we pay for it. More people needing more care is expensive. Figuring out how to not pay for that care isn’t going to reduce the health spending, figuring out how to pay fairly for the right care at the right time in the right setting though just might.