By ALAN SCHWARTZ and SAUL WEINER

In their 1993 book, Reinventing Government, David Osborne and Ted Gaebler entitled a section “what gets measured gets done.” Unfortunately, when it comes to improving health care quality, safety, and costs, we often fail to observe the real work of care, and miss the chance to get it done better. To make a real difference, we need to begin measuring care when and where it happens – behind the curtain.

Why We Must Directly Observe Patient Care

For the last 10 years, our work in research and quality improvement has used concealed audiorecorders to capture what actually happens during patient-physician encounters, and to provide feedback to physicians about their performance. Much of our focus has been on demonstrating the importance of appreciating the patient’s life context and showing how encounters in which physicians elicit patient context and incorporate it into care planning have better health care outcomes and lower costs from inappropriate care. We’ve found such contextual factors are relevant to health care in two-thirds of encounters, that physicians ask about them less than a third of the time, and when they are discovered, they are incorporated into the plan less than 60% of the time. Contextual errors—inappropriate care due to failure to contextualize—are pervasive.

Records Don’t Record

Only direct observation of care reveals these errors. The medical record, currently the source of most data in performance improvement, does not and cannot identify mistakes that the physician doesn’t already recognize. The medical record, at best, shows that the physician rendered the care they believe the patient needed, which can be the “right” care for the wrong patient.

Moreover, studies comparing the accuracy of the medical record to the care actually provided suggest both that care is documented that is not provided and that care is provided that is not documented.

Patient care surveys, another source of quality improvement data, are helpful but also limited. Patients know the complexities of their own lives, but frequently don’t understand how that context relates to their health care. A patient may be stretching out an expensive daily asthma inhaler to reduce the cost of refilling the prescription without understanding that the medication’s effect depends on daily use. Just as the physician doesn’t know the problem to treat without exploring the patient’s life, the patient doesn’t know that his problem is one of access and knowledge, rather than lung function.

Mystery Patients and Focused Improvement

Another important technique for direct observation of care is the use of the unannounced standardized patient, an actor trained to visit a physician incognito and portray a particular patient case with high fidelity. This “mystery patient” approach, like the use of “mystery shoppers” to improve customer service or evaluate racial disparities in customer treatment, allows a practice to manipulate features of the patient presentations and evaluate performance while controlling for case mix. In our largest such study, physicians planned appropriate care 73% of the time when the case did not include a complicating medical or contextual factor, 38% of the time when the case included a complicating medical factor, and only 22% of the time when the case included a complicating contextual factor. In a recent quality improvement project, we trained actors to present as diabetic patients and record whether the physician conducted a proper diabetic foot exam; comparing the directly observed care to the charted care reveals preventable errors in care, charting, and billing. The use of unannounced standardized patients, particularly with the prior consent of physicians to receive such visits (at an unspecified future time), can meet ethical standards for research, as well as quality improvement.

How to Move Forward

Although much can be improved in medical education settings, directly observing care is an especially important step toward measuring physician performance in the natural setting of practice. Providing ongoing audit and feedback to physicians about their performance can improve contextualized care (Weiner, et al. 2015 June. The Joint Commission Journal on Quality and Patient Safety) and suggest specific quality concerns that can be tested in detail with unannounced standardized patient programs. Practices may adopt direct observation as part of their commitment to improve quality improvement and patient safety; payers and regulators may also reap the benefits of reduced costs by encouraging these efforts through performance incentives.

Alan Schwartz, PhD, and Saul Weiner, MD, are co-authors of the forthcoming book Listening for What Matters: Avoiding Contextual Errors in Health Care (Oxford, January 2016).  They write about contextualized care and direct observation at http://contextualizingcare.org.