Anatomy of a Healthcare Quality Metric

Screen Shot 2015-06-19 at 8.24.15 AM

In the future, doctors who provide better healthcare will be paid more. When a doctor gives good care, she will get credit. For factors out of that doctor’s control, she won’t be penalized. The patient, too, will be rewarded for taking care of his own health. In short, payments will align with good care, and good care will become more common.

This is the promise of value-based care, which is coming, according to almost everyone. Medicare is pushing it. Private payers are preparing for it.Top providers are tooling up.

And yet, the question lingers — how exactly do we measure quality? Today quality measurement is rigid, periodic, and manual. Here’s a peek behind the curtain of what we measure today — and what’s possible tomorrow.

Quality Decomposed: Three Examples

The Physician Quality Reporting System (PQRS)

Snapshot: PQRS is a clinical quality reporting requirement for all Medicare Part B Physicians. The are 254 measures total, and providers report on a subset of these. Every provider that takes Medicare Part B is liable for up to 6% of Medicare payments in penalties and qualifies for up to 6% in rewards based on performance.

Cost of reporting: Some practices report via claims or through their EHR, which costs no money (though often a lot of time). Medicare is now incentivizing practices to report through a CMS certified registry, which costs between $100 and $600. Consulting for PQRS reporting runs a few hundred dollars per hour, and a large practice may spend upward of $25,000 on consulting for PQRS.

What it looks like: PQRS001 — Diabetes — Hemoglobin A1c Poor Control

This metric is asking practices to calculate how many patients with a diagnosis of diabetes who were between 18 and 75 on the date of their visit either 1) did not have an A1c test, or 2) had an A1c test greater than 9.

The good and the bad: To start, 9% is quite high for A1c. You don’t want patients up at 9%, so it’s good to know if a lot of patients are. One practice administrator interviewed told the story of how A1c performance metrics drove her clinic to implement diabetes counseling — 30 minute appointments with nurses to go through lifestyle questions and create an eating and exercise plan for each diabetic patient. The practice has seen a marked drop in A1c levels — and better performance on the diabetes metrics.

But the 9% metric is troublesome. What if a patient’s HbA1c was at 9.3% last month and has steadily come down to 9.1%? What if it were rising really quickly but hadn’t reached 9%? What if the doctor ordered an A1c test like she was supposed to, but the patient never filled it? What if the patient was diagnosed late in the year and didn’t have a chance to get the test until after the reporting period? Maybe the doctor wrote in the notes that the patient got an A1c test from another provider, but that doesn’t get pulled out for the quality report. None of these cases are counted, and the doctor gets dinged.

Does this seem frustrating or confusing? Only 253 PQRS measures to go.

Meaningful Use (MU)

Snapshot: MU is a certification and incentive program that encourages Medicare providers to use electronic health records systems more effectively. It measures how doctors use EHRs to engage patients, increase healthcare coordination, maintain security and ultimately improve healthcare quality and safety. Meaningful Use Stage II is made up of 17 core measures for eligible professionals, and 16 measures for hospitals. This stage is currently being rolled out after repeated delays. The penalty for failing to comply with MU in 2015 is between 1% and 2% of Medicare reimbursements. Stage III is on the horizon, but unclear if and when it will hit.

Cost of reporting: A Meaningful Use certified EHR system can be free, for a cloud-based system like Practice Fusion, or up to hundreds of thousands of dollars for a traditional on-premise system. Many larger practices have one or more full time employees dedicated to meaningful use implementation and attestation. Some smaller practices choose not to comply and take the 1% penalty because the cost of compliance is too high.

What it looks like: Meaningful Use Measure 2 — Generate and transmit permissible prescriptions electronically

The good and the bad: In some cases, e-Prescribing has been shown to reduce prescribing errors as much as 85%, as well as reducing costs and increasing efficiency. In other cases, e-prescribing has caused unprecedented errors, such as prescribing 38 times the recommended dose of antibiotics.

Even when it works, today the rigidity of the e-prescribing system ignores current patient and physician preferences. For example, this draconian fine print in the e-prescribing metric documentation:

Instances where patients specifically request a paper prescription may not be excluded from the denominator of this measure. The denominator includes all prescriptions written by the EP during the EHR reporting period.

One physician explained that she likes to provide patients with a paper prescription in addition to sending it electronically. She has seen that for patients, having the physical paper helps remind patients to pick up their prescription, and also she’s never entirely sure the electronic order went through. Now her Meaningful Use certified Epic system will not let her print a prescription, ever.

Consumer Assessment of Healthcare Providers and Systems (CAHPS)

Snapshot: CAHPS is a set of patient surveys that evaluate patient experiences with health care. The results are available to the public and are used by Medicare to determine some payments. Each year the CAHPS survey database aggregates nearly 100,000 patient responses across surveys designed for inpatient and outpatient across a number of specialties.

Cost of reporting: running a survey at a hospital through a qualified vendor costs between $2K and $6K, internal staff time not included. Surveys are usually conducted once a year.

What it looks like: CAHPS 2015 Hospital Survey — Page 1

The good and the bad: It makes good sense to ask patients what they thought of their care. The problem is that as patients our broader impression of the experience tends to matter more than specific quality indicators. A study at Johns Hopkins found little relationship between a hospital’s patient satisfaction scores and most other quality ratings. One researcher warned that patient satisfaction metrics are “going to mislead patients because they’re going to think the hospital with the best lobby and the best parking and customer service is going to have the best heart surgery.”

A Living Quality System

Looking at the inner workings of quality measurement today, it’s obvious that it won’t do. Today we measure just a few metrics which barely represent quality, and already the costs are putting providers out of business and causing frustration enough to make some doctors retire early. The future will not look like this. Here are some predictions about what will change:

  • We won’t track a single indicator, like Hemoglobin A1c level, once a year. We will instead track many indicators constantly using an array of monitoring and tracking devices like Google’s glucose monitoring contact lens or Scanadu.
  • Rather than relying on the results of a single lab test to indicate quality of care, we will develop measures that adapt to specific patient circumstances. Trends like improvement and decline will be counted. There will be risk adjustment for very sick patients so that they are measured against metrics that fit their circumstances.
  • We will build a more sophisticated feedback loop between measures and outcomes so that the system can rapidly evolve toward measures that matter to patients rather than those that are easiest to measure or have the most lobbyists behind them. The Institute of Medicine’s calls this aLearning Healthcare System, one that “is designed to generate and apply the best evidence for the collaborative healthcare choices of each patient and provider,” This learning system depends on collection and analysis of clinical data.

We are in the early days of robust quality measurement in healthcare, and now is a necessary but painful pilot period. Metrics are imperfect, and we should not expect or allow the current system to become entrenched.

At the same time, today’s metrics are the foundation for our future value-based system, and are beginning to drive real change in medical practice. Now it’s our job to hone them, evolve them, and work for the future state where they truly recognize and reward good healthcare.

Rachel Katz is the co-founder of Able Health and a regular THCB contributor. In her past life, she led a joint project at McKinsey & Co and the World Economic Forum showing how norms and values are shifting in the digital age. She is the author of the forthcoming “Long Haul: An American Girl, Chinese Truckers, and Unexpected Lessons from the Road”

6 replies »

  1. Thanks, Millenson. I’m optimistic about the future of quality measurement, too. It seems that more and more metrics development today is happening at medical societies which have created certified data registries (for example the ACC’s registry: http://cvquality.acc.org/NCDR-Home/Research.aspx). Maybe qualified clinical data registries will become a center of gravity for researching and testing quality metrics. I’m interested to hear your thoughts on this – who do you think will/should be responsible for metrics development in an ideal world?

  2. Thanks for the comment – I agree that today’s metrics are mostly process indicators. I hope/expect in the future we will move toward outcomes measurement as the systems becomes more sophisticated. I’ve had my eye on ICHOM (http://www.ichom.org/), an early effort to develop and evangelize a measurement system focusing on “results that matter to patients.” If we believe that healthcare needs quality measurement (which I do), then it should evolve to look more like what ICHOM is developing, less like the process metrics we have today.

  3. Please tell us how “experts on making American health care better, safer and more patient-centered” were dragged “kicking and screaming into a situation of accountability.”

  4. Nice article, Rachel. What it shows it that when you drag any group kicking and screaming into a situation of accountability, they’ll eventually pull up their socks and, through the magic of voluntarism, improve what was imposed on them. I look forward to the next generation of physicians forgetting that a lack of measures was ever an option and instead concentrate on inventing sensitive ones.