By David Shaywitz, MD

Quality is the new watchword in healthcare; it’s what we seek – and increasingly, what we try to measure.  Medications, devices, care delivery, hospital services – all are now scrutinized as we seek to gauge their benefit, and justify their cost.

The idea of using metrics to evaluate quality make sense, but only if we can trust the metrics themselves.  Otherwise, we risk becoming party to an updated version of craniometry, systematized false-precision that focuses on easily-measurable parameters (such as head circumference) that may not represent meaningful proxies for the assessments we’re really after (i.e. intelligence).

The good news is that the science of testing, of developing evaluation instruments, has improved over time.  We’re now better able to recognize the qualities and properties of good tests – and to identify where they’re likely to fall short.

We’re also getting more comfortable with demanding robust evaluation instruments.  For example, the FDA’s approach to patient-reported outcomes places exceptional (and appropriate) emphasis on the assessment tool chosen, and requires that it demonstrates the appropriate properties before relying on its results.

Unfortunately, one critically important area within our healthcare system that seems to have escaped such careful review is the way the competence of care providers is typically assessed and certified.

Whether you are an X-ray technician, a physical therapist, a registered nurse, or a transplant surgeon, you are required to pass through a gauntlet of costly certification exams.  These tests, already significant, are assuming an even greater importance as the healthcare system increasingly looks to them as proxies for quality.  Certification can be required for employment and for admission privileges, and frequently impacts the reimbursement rate for healthcare providers.

All this makes complete sense – provided the certification tests themselves are sound.

Unfortunately, the world of healthcare worker certification remains a bit like the wild west, as medical organizations and professional societies approach certification testing with profoundly different degrees of rigor — and generally little-to-no transparency.

Continue reading…