Are Mystery Shoppers Such a Bad Idea for Health Care Quality Improvement?

Are Mystery Shoppers Such a Bad Idea for Health Care Quality Improvement?

35
SHARE

The decision by the Obama administration to employ “mystery shoppers” to pose as patients to see how difficult it is to get an appointment with a physician has sparked criticism from physicians. However, access to primary care physicians is a very real public policy issue that needs to be understood if we are to successfully care for the more than 30 million Americans who receive coverage under the Affordable Care Act.Is the use of “mystery shoppers” a bad idea?

Dr. Raymond Scalettar certainly thinks it is a bad idea. “I don’t like the idea of the government snooping. It’s a pernicious practice – Big Brother tactics, which should be opposed.”

Dr. George Petruncio says, “This is not the way to build trust in government. Why should I trust someone who does not correctly identify himself.”

Westby Fisher, MD writes in his blog: “When information gathering trumps patient care – particularly fictitious care – we’ve got a problem. Is this a new quality standard we can expect from our new government health care initiative?  Just like scam-artists that phish for unsuspecting people’s financial information online, governmental appointment phishing should not be tolerated in any way, shape, or form. It is fraud – plain and simple.”

Several physicians on twitter retweeted Dr. Fisher’s blog post and indicated they agreed with his analysis.

 

A mystery shopper is a quality improvement tool that has been used in the retail and hotel industry for decades. In 2004 the health care industry accounted for 2% of mystery shopper revenue, but the use in health care is increasing.

In health care, mystery shoppers have been used to provide feedback on every step in the patient experience: making an appointment, environment of the waiting room, encounter with nurses and office staff, and visit with the physician. There has been an increased interest in mystery shoppers by health care providers since CMS started posting patient satisfaction information on hospitals as part of the Website Hospital Compare.

The use of mystery shoppers has been reported to lead to better patient flow and improved wait times, extension of office hours, improved telephone etiquette, better physician communication with patients, and more time with patients answering questions about surgery.

The AMA Council on Ethical and Judicial Affairs studied the mystery shopper question and concluded that a sound program would include:

1) The places that unannounced visits will affect should know about the program.

2) The information should be used for improvement and not punitive actions.

3) Mystery shoppers should not be the sole source of data for evaluating clinical performance

4) The program should not adversely affect access to medical care by legitimate patients.

I disagree with my colleagues that a properly planned and implemented mystery shopper program is a bad idea for trying to improve health care. For far too long, we in medicine have been too arrogant to learn lessons from other industries that improve quality. I think we need all the help we can get to take better care of patients.

Kent Bottles, MD, is past-Vice President and Chief Medical Officer of Iowa Health System (a $2 billionhealth care organization with 23 hospitals). He was responsible for the day-to-day operations of a large education and research organization in Michigan prior to his work with in Iowa with IHS. Kent blogs at Kent Bottles Private Views.

Leave a Reply

35 Comments on "Are Mystery Shoppers Such a Bad Idea for Health Care Quality Improvement?"


Guest
Lynn
Jun 28, 2011

A recent study published in the New England Journal of Medicine by Bisgaier and Rhodes, “Auditing Access to Specialty Care for Children with Public Insurance,” indicates that children covered by Medicaid are less likely to get an appointment with a pediatric specialists than a child whose parents have private insurance. It also showed that getting an appointment with a pediatric specialist wasn’t easy either.

I suspect that a secret shopper study for adults will show similar results. The less attractive the third party coverage the harder it is to get an appointment. The ugly truth is that public insurance coverage is often just a license for a patient to hunt for care. So once we’ve proved this, what are we going to do about it? Increase physician compensation? I don’t think so.

The number of primary care physicians is in the short run fixed, since it takes 8 years to produce one. It’s only prudent business practice to restrict the number of patients or visits and to limit those visit to patients who compensate providers the highest amount or with least restrictions/ hassle factors. There are taxes and salaries to be paid not to mention the payments on the new EHR software….. Physicians see more than 25 or 30 patients a day burn out.

Develop a new business model of care — the medical home? Without enough practitioners we might as well call it a medical shack.

A new business model depends on the incentives of how we pay for care. Until a new payment model is devised that is fair to physicians and patients, barriers to access will remain. Legal actions against physicians who don’t accept new patients is counterproductive and will drive more and more physicians to turn to concierge practices where only an American Express card is accepted. Is that the model we want?

Spending two years to prove what we all know is going on doesn’t resolve the problem of too few primary care providers (or pediatric specialists). If we value something we count it and we pay for it. More people needing more care is expensive. Figuring out how to not pay for that care isn’t going to reduce the health spending, figuring out how to pay fairly for the right care at the right time in the right setting though just might.

Guest

What’s missing from the conversation of using mystery shoppers to evaluate access is identifying the question this process is designed to answer. It can only evaluate access for new patients to a primary care practice, which is likely to differ from access for established patients.

Guest
tim
Jun 28, 2011

Every question in the form of “Is X a good idea?” has two types of answers: one for if the government does it, all the others for when the agent is not the government. Many, many actions are sensible except when the government does them.

At the minimum, sophisticated observers build into their analysis this metaphysical truth: any entity which is, by definition, insulated from normal feedback loops is presumed to be corrupt.

Mystery shoppers from the government will become the gestapo, no matter their larval form.

Guest
Jun 28, 2011

I not to thrilled with the federal government doing anything secretive against ordinary citizens including law abiding physician practitioners

Guest
Jun 28, 2011

I think the main goal for mystery shopper is to improve the quality for the patients, which doesn’t seem like a bad thing and I agree with it.

Guest
pcp
Jun 28, 2011

Sillly, silly study.

We already know we don’t have enough primary care docs. We already know many docs don’t see Medicare patients. We know even more don’t see Medicaid patients. What new info is this little study going to give us?

Next up: the government hires mystery naturalists to determine if bears defecate in the woods.

Guest
Jun 28, 2011

It was an ARRA earmark expenditure from the U.S. Department For The In-Depth Study of Blinding Glimpses of the Obvious.

Guest
Craig "Quack" Vickstrom, M.D.
Jun 30, 2011

ROFLMAO! This whole mystery-shopper thing is a sad indication of just how out-of-touch and clueless these Beltway ppl are.

Guest
James
Jun 28, 2011

A more reliable feedback mechanism is surveying the patients themselves. Then interview others with insights into access, such as your disease and case management services vendors and/or your managed care organizations (Medicaid/CHIP uses a lot of them). The latter have network adequacy provisions in their contracts, so they should be reporting access-related stats on this metric anyway — or you should be asking for them. Also, track closely any complaints received regarding an inability to get appointments. Then follow up to see if they were fully resolved. You need to know if REAL patients with REAL Medicare and Medicaid coverage can’t get an appointment. Another item to examine is whether the rates vary depending on whether it is a FFS or HMO model (Medicaid may use both in the same state).

You can also gain insights from looking into your ER frequent flyer populations, protentially preventable hospitalizations, and potentially preventable readmissions (may flag a lack of post-discharge appointment opportunities). None of these activities requires hiring a new vendor or any dishonesty with the doctor or their staff. All can also typically be done with existing administrative (claims and encounters) data and current vendor relationships.

What would be most useful would be insights into what has actually worked to increase the availability of appointments (both primary care and specialty). Was it reimbursement increases? If so, how much? Was it personal outreach to the physician to recruit? Was the option to accept a limited number of new Medicare or Medicaid cases explored? Were physicians who didn’t take Medicaid or Medicare asked why? Did they have a problem getting paid? Did they think it was too much hassle (too many prior auth requests? too many rejected claims? Etc.?) Did they get a lot of client no-shows? Was getting any sort of compliance with the treatment plan by the patient overly difficult. Were there language or medical literacy issues?

Identifying that a provider doesn’t take Medicare or Medicaid or that a patient can’t get a timely appointment doesn’t actually solve anything, in and of itself. You have to get to the “why” part of it all. Any possible solutions will flow from there.

If anyone does have evidence regarding strategies that worked to improve access, I would welcome a THCB article posted on this topic.

Guest
Jun 28, 2011

“A mystery shopper is a quality improvement tool that has been used in the retail and hotel industry for decades.”

But they’re employed or under contract to the retail outlet or the hotel, not the government. Big difference. I think there would be big legal problems with either CMS or an insurer sending mystery shoppers into health care facilities under false pretences.

Guest
MD as HELL
Jun 28, 2011

I have never joined the AMA and now I know why. They are a bunch of flaming idiots who just want to be liked by the government.

There always have been mystery shoppers. They will only learn what is already known: Mcare and Mcaid and Tricare are not widely desired by primary care doctors. That is not news. Many practices will flatly tell callers they are not taking any new Mcare patients. That is not illegal. Mystery shoppers are unnecessary for that purpose.

So what is their real purpose?

J. Edgar Hoover would be proud.

Guest
Craig "Quack" Vickstrom, M.D.
Jun 30, 2011

The AMA is also totally in the pockets of the specialists, and exists to shaft primary care.

Guest
Jun 28, 2011

“A mystery shopper is a quality improvement tool that has been used in the retail and hotel industry for decades.”
___

And, everyone knows that “shopping” for health care is no different than shopping for shoes or flat panel TVs. Just ask John Mackey.

Guest
steve
Jun 28, 2011

Sometimes what everyone knows is not true. If true, it needs to be quantified. Why should we just guess? I fail to see what people are worried about.

Steve

Guest
James
Jun 28, 2011

“I fail to see what people are worried about.”

For one thing, we are borrowing 40 cents on the dollar right now – and this was a wholly uneeded expense. Medicare and Medicaid could get this same data from multiple existing sources, as was already noted above. Just SURVEY THE PATIENTS for gosh sakes! There would be no need to have people be dishonest with doctors and their staff.

Guest
steve
Jun 28, 2011

Patient surveys have their own problems, and if you do them correctly, they cost just as much. Aaron Carroll links to a new article from the Archives of Internal Medicine which claims much higher acceptance of new Medicare patients than is commonly thought. Lower for private patients than one would expect. We cannot make good decisions w/o good data. How else do we get it? The study, as I have seen it described, should resolve a number of problems with methodology issues.

http://theincidentaleconomist.com/wordpress/doctors-arent-accepting-new-patients-with-private-insurance-either/

Steve

Guest
Jun 28, 2011

What do we want to achieve? Verify access for medicaid patients? If there is a problem, it just indicates that there is a mismatch of reimbursement between medicaid and other payors.

Clinical quality is very hard to measure. There are some surrogate parameters that medicare is already measuring (beta blockers for MI, time to ABx), more in depth assessment could probably done via independent peer review only.

Patient surveys are heavily biased by convenience factors such as parking, pleasantness of facility etc, and many patients are easily seduced by perceived “cutting edge” nonsense (as are many physicians). Even communication assessments can be flawed when patient’s don’t like the provider’s message.

I practice in a competitive metropolitan area (and practiced in an even more competitive one before). Patient satisfaction centeredness brings you quick access (too quick access drives up costs), very polite, pleasant docs (a good thing, but not when docs feel they have to take patient requests for testing, without asking because that could make ’em unhappy), shiny facilities with marble, valet parking and birthing suites, and stupid direct to consumer advertising for the latest technology … it’s good for the wealthy if they want pay for it. The bottom half needs just reasonable, affordable healthcare that is not doing diagnostic& therapeutic overkill.

Guest
James
Jun 28, 2011

“If there is a problem, it just indicates that there is a mismatch of reimbursement between medicaid and other payors.”

In general you are correct, but it can also indicate a lot of other things. It can indicate that the provider (and we are talking more broadly than just docs) finds the hassles of Medicaid to be too cumbersome. Maybe they dislike the prior auth procedures, feel too many claims are rejected, that payments are late, or the clients are hard to deal with due to no-shows, non-compliance, language or medical literacy issues, a prevelance of MH/SA (mental health/substance abuse) issues above the rest of their panel, that clients lose coverage too often, etc.

Patient surveys can indeed be flawed if you measure subjective (even nebulous) terms like satisfaction. However. asking a new Medicaid patient if they were able to get PCP after joining the program isn’t rocket science. it is an objective question, with a binary answer. Ditto for asking the ER frequent flyer with asthma why they show no claims for a PCP visit. Target the queries to get right to the problems.

You call for good basic health care is one of merit, by the way.

Guest
James
Jun 28, 2011

The root problem is that there is dishonesty at work. How does one build trust with providers when people on your employ will CALL THEM UP AND LIE TO THEM.

Next, I simply reject the contention that ASKING THE CLIENTS if they are having problems scheduling appointments or asking doctors if they have panel openings costs as much as hiring a survey firm. Seriously, the Medicaid or Medicare agency is already communicating with both parties. If it through an MCO model (Medicaid Managed Care), then the MCO is contractually obligated to provide an adequate network. If clients can’t get access, they are breaching their contracts. Also, if it is a Medicaid agency, they are taking complaints if a client can’t get access. Simply track the complaints and look for patterns with better data collection at the time of the call. Existing data and existing client and vendor relationships can – and should – be what is used to gather usable data on accessgaps. Sorry, but this is still a complete waste of cash.

Guest
Jun 29, 2011

http://thehill.com/blogs/healthwatch/other/168905-hhs-cancels-qmystery-shopperq-proposal Late Tuesday night the Obama Administration and HHS decided to withdraw their proposal to use mystery shoppers to assess availability of physician appointment slots. We really do have a hard time holding anybody accountable in American health care.

Guest
MD as HELL
Jun 29, 2011

They will still deploy their goons

Guest
Jun 29, 2011

http://ow.ly/5sLWa How can HHS say with a straight face that politics had nothing to do with canceling the mystery shopper program? Come on