As reported in The New York Times, thirty-seven of Newsweek’s top 50 high schools have selective admission standards, thereby enrolling the cream of the eighth grade crop. That means that when these high scoring eighth graders reach eleventh grade, they’ll be high scoring eleventh graders, helping the school move up the Newsweek rankings. These selective admission schools simply have to avoid screwing up their talented students.
That’s no way to determine how good a school is. The measure of a good education should be to assess how well students did in that school compared to how they would have been predicted to do if they had gone to other schools.
Imagine two liver transplant programs, one whose patients experience 90% survival in the year following their transplant and the other whose patients experience only a 75% survival rate. Based on that information, the former hospital looks like the place to go when your liver fails. But aren’t you curious about the kind of patients that receive care in these two hospitals? Wouldn’t you want to know whether that first hospital was padding its statistics by selectively transplanting relatively healthy patients?
When hospitals are judged by patient outcomes, savvy hospital administrators find ways to bolster their statistics. That’s why, according to a 2005 JAMA article, when New York State began reporting mortality rates and complication rates for patients undergoing cardiac surgery, hospitals in that state began to game the system. They found ways to avoid patients who were less likely to survive after surgery, minority patients, for example, or patients with lots of other illnesses, what doctors call co-morbidities.
New transplant programs, eager to qualify for Medicare reimbursement, work hard to bolster their transplant survival statistics, because Medicare looks for proof of success before counting a program as being eligible for reimbursement. The best way to achieve good survival statistics of course is to transplant the healthiest candidates. Heck, if you really want good survival rates, you should transplant me—I’m perfectly healthy (aside from a few worn-out joints)!
For the last two years, I have taught an undergraduate health policy course at Duke University with the assistance of Public Policy teaching assistants who have interests in education policy. These TAs often remark on the parallels between education policy and health policy. The challenges of measuring health care quality, for example, closely parallel the difficulties of assessing the quality of an educational institution. In healthcare, we “risk adjust” our measures, to even out the playing field between hospitals that care for otherwise different populations. These risk adjustments are not perfect by any means. But researchers are slowly improving their statistical measures.
Education policy needs to aggressively adopt the same kind of risk adjustment measures, if we hope to identify which high schools are truly doing the best job of educating their students and preparing them for the future.
Peter Ubel is a physician, behavioral scientist and author of Pricing Life: Why It’s Time for Health Care Rationing and Free Market Madness. He teaches business and public policy at Duke University. Peter’s new book, Critical Decisions will be available in the fall of 2012. You can follow him on his personal blog.