British Petroleum’s Wellness Program is Spewing Invalidity

British Petroleum’s Wellness Program is Spewing Invalidity

31
SHARE

A critical observation in Cracking Health Costs is you need not “challenge the data” to invalidate claims that wellness saves money.  Instead, you can simply read the data as presented.  You’ll find it usually invalidates itself.

Nowhere is that more true than in a study published this month by Mercer, Staywell and British Petroleum (“BP America”) in the Journal of Occupational and Environmental Medicine (JOEM).   As we’ll demonstrate, the results completely contradict Staywell’s own statements, and are also mathematically impossible.  Indeed, Mercer was a wise partner choice by BP America because their validations are often unconstrained by the limits of possibility.   For instance, they validated massive savings both for infants in a North Carolina Medicaid program that did not enroll infants, and for a Georgia Medicaid disease management program that did not manage diseases, at least according to the FBI.

Along those lines, let’s see what happens when one compares the JOEM conclusion — that the Staywell wellness program for BP America achieved almost $20,000,000 in savings on 20,343 BP participants after only two short years – to the limits of possibility.

It turns out this overall savings claim of $1,000/person would require completely wiping out wellness-sensitive medical events (heart attacks, diabetes events etc.) not just on those 20,000+ people, but also on perhaps 40,000 of their closest friends.  The authors elected not to disclose the change in wellness-sensitive medical events across the entire eligible population, perhaps because they were embarrassed by the size of the decline, if indeed those events declined at all.

The authors went a step farther and decided not to even acknowledge that a wellness program should reduce wellness-sensitive medical events, a perfectly logical nexus not unlike saying that an antihyperlipidemic drug should reduce cholesterol levels or an asthma rescue drug should improve breathing, or sunblock should prevent sunburn.

It’s not just the two of us who think logic should prevail in connecting wellness initiatives to wellness events.   Health Affairs laid them out too, in a seminal 2013 study that Mercer and Staywell didn’t footnote despite being the most recent scholarly article on exactly this same topic.  They also selectively omitted other key citations.  The equally seminal RAND study – which concludes that wellness programs produce no two-year population-wide savings – also went unacknowledged.

Curiously biometric screens were noted as one of the sources of BP’s alleged savings, but this once again turns out to breach the wall of possibility:   another – also unacknowledged — Health Affairs article showed that it is mathematically impossible not to lose money on biometric screens, which is why government agencies, expert panels and books warn against using them.  (They can and do also harm patients, which is perhaps why most people only participate when threatened with large penalties, as was the case with BP, where 30% of their employees elected to forfeit $1200, to stay out of their program.)

These selective citation omissions are indicative of the wellness industry researchers’ distaste for facts.  This is not to say that wellness researchers are stupid.  Quite the opposite: they are smart enough to realize that facts are their worst nightmare.

The Data that Didn’t Bark in the Nighttime

And in this case, the most troubling fact is right in their own study, albeit buried deep enough (page eight) that the JOEM peer reviewers may not have noticed it.  That is the percentage of people whose risk factors actually improved:  5.6%.  As modest as this figure is, it actually excludes the people most likely to get worse– those who dropped out of the program and those who wanted nothing to do with it in the first place.

Understandably, the study authors decided not to draw attention to the simple division of the savings ($19,700,000) into the number of people who even theoretically could have avoided wellness-sensitive medical events (5.6% of 20,343, or 1,139 people), because that division would have yielded a whopping $17,300 in savings per person whose risks declined.

And once, again, the limits of possibility don’t stand a chance against Mercer and Staywell, for three reasons.  $17,300 is:

  1. Almost three times what a company spends in total on the average covered person`.

  2. At least 30 times what a company spends on wellness-sensitive medical events for the average covered person.

  3. More than 100 times what the lead Staywell author says on Staywell’s own website is possible.

This assumes there were any savings at all, which we’ll never know because recall they failed to disclose the wellness-sensitive medical event tally.

Lessons for Wellness Researchers

So, what can be learned from this valuable addition to our growing library of teaching tools for population health management, other than how easy it is for vendors and consultants to convince their HR clients in large corporations that they are saving massive sums of money through wellness?

First, the most sophisticated-sounding study designs in the world – and this one looks at least six figures worth of sophistication – can’t ignore the question:  “Is this result even mathematically possible?”

Second, those expensive “matched control” study designs are worthless anyway when comparing actively motivated participants to unmotivated non-participants.  You can match demographics but you can’t match the difference in mindset between (for example) smokers who want to quit and smokers who don’t.  Health Fitness Corporation showed that would-be participants significantly outperform non-participants even without a program.

Third, researchers need to stop attributing all good things to wellness and instead employ a basic biostatistical tenet, which is that the outcome should bear some relationship to the intervention.  In the case of wellness, that means a program designed to reduce wellness-sensitive risks should have its greatest if not only effect in avoidance of medical events of a wellness-sensitive nature.

Fourth, they need to understand the relationship between risk factor reduction and cost savings.  As Why Nobody Believes the Numbers shows mathematically, costs decline about a tenth as much as risk factors decline, meaning that BP could attribute roughly a 0.56% decline in costs to the 5.6% risk factor decline.  This 0.56% gross decline would be offset by more preventive physician visits, more preventive drugs, the cost of the program and incentives.  It also assumes that the 5.6% itself figure is real, which it almost certainly isn’t because…

…Fifth, this industry needs to stop ignoring the overwhelming preponderance of facts that don’t support its belief system.  In particular, it needs to stop citing the single company (Johnson & Johnson) which has a strong wellness culture that few others have duplicated , and instead look at RAND and (for example) IBM, Dow Chemical or Salt Lake County to see a much more realistic view of what companies can accomplish.   Wellness industry leaders specialize in distorting fundamental scientific and analytic principles to meet their own needs.  Increasingly, however, health policy experts are calling them on this, which means that there is hope for introducing integrity into the wellness outcomes reporting process.

Along with their distaste for facts is a distaste for arithmetic.  Page eight of the JOEM paper asserts that “there are no industry standards on how to calculate ROI…”  They could easily have added:  “…because most vendors, including us, refuse to disclose the rate of wellness-sensitive medical events, which would be the obvious choice as an industry standard if we were able to actually reduce these events.” Instead the vendor association uses an invalid consensus methodology, making wellness/disease management the only industry where mathematical proof is trumped by popular vote.

Finally, industry supporters need to stop citing the single article in a first-tier journal with a positive finding about wellness’s impact on health spending.  That article is now approaching its fourth birthday, it was a meta-analysis of studies mostly drawn from third-tier journals that among them had never published an article that found anything other than positive results from wellness (there is even a term for this — “publication bias”), and even its authors no longer seriously defend it.  (Reflecting the dearth of more recent positive press in first-tier journals, a leading wellness proponent recently wrote that this article “should be cited much more frequently by health promotion and wellness professionals.)

This industry needs to acknowledge that almost all credible recent literature – ours or anyone else’s –finds largely failure, and then develop or identify some best practices.   Hence we once again invite people to identify a single wellness vendor that saves money by reducing population-wide wellness-sensitive medical events, for us to highlight in a future column.

Al Lewis is the author of Why Nobody Believes the Numbers, co-author of Cracking Health Costs: How to Cut Your Company’s Health Costs and Provide Employees Better Care, and president of the Disease Management Purchasing Consortium.

Vik Khanna is a St. Louis-based independent health consultant with extensive experience in managed care and wellness.  An iconoclast to the core, he is the author of the Khanna On Health Blog.  He is also the Wellness Editor-At-Large for THCB.

Leave a Reply

31 Comments on "British Petroleum’s Wellness Program is Spewing Invalidity"


Guest
R Quinn
Jul 10, 2013

Excellent article. I have been saying the same thing for years without any statistical proof, just common sense and logic. I once managed a wellness program and spent over $2 million a year doing it. Nobody asked about hard savings or proof of savings (there weren’t any), but nevertheless it was instinctively and politically a good thing to do so the money was spent year after year and still is to this day. Employees think they save on their premium by participating. In fact, the supposed premium savings are fictitious because the discount they receive was added to the premium. Wellness in the corporate setting is a house of cards that nobody has the guts to blow at.

Guest
Jul 10, 2013

R Quinn, thanks for your note and your validation of our argument. We are pleased to have you on board.

Guest
Sam Lippe
Jul 10, 2013

When I read these essays on wellness, they all seem so cut-and-dried — I’d like to see a rebuttal by the named parties

Guest
Jul 10, 2013

Sam: we, too, would like to read rebuttals. We relish the rhetorical brawl. But, regardless of whether we write for THCB, Health Affairs, the Wall Street Journal, (or other pending outlets), the industry is silent, save for the few wellness acolytes who call us names on LinkedIn. Debate is a competitive process, and when the opposition not only refuses to show up and engage, but they won’t even defend the claims they’ve already made in supposedly reputable peer-reviewed journals, it tells you an lot about both their courage and their integrity.

Al and I are not done, not as long as they keep claiming impossible savings and improvements in health and using those same false claims to dupe (sometimes unwitting, but sometimes complicit) employers and to ramrod employees into things that they don’t want to do (such as take HRAs and submit to biometric screenings). We aren’t done by a long shot.

Guest
Mary
Jul 11, 2013

You apparently haven’t read Al’s book, which is surprising because I’ve seen your comments on his postings before. He says the wellness industry never resonds or even acknowledges his things, in order not to draw attention to them. Doubt stay well and mercer have even told BP

Guest
td
Jul 11, 2013

Agree with Mary – the vendors are not disposed to let their clients know the data has issues…too much $$$ at stake. Responsible consultants own up to their mistakes and fix their methods.

Guest
Lia Spiliotes
Jul 11, 2013

These stories of yours are always hilarious, how you call them idiots without calling them idiots just by quoting their own data. What does BP think of this? How about Mercer or the vendor? Isn’t there anyone who is doing this right? I’d love to read a good-news story even if the bad news ones make for better copy. Al is the definitive “go to” source to spot the insanity and just plain idiocy of claims a described above.

Guest
Jul 11, 2013

Yes, like you and like Diogenes, we are also looking for an honest story of savings. It’s a good question and you are not the first one to ask it.

Specifcally, I don’t want to imply that all vendors and consultants make up outcomes, not by any stretch. 80% are honest…and don’t release their outcomes because they know they don’t save money. There are many reasons to have a culture of wellness as an end in itself, but reducing healthcare spending is not one of them.

And thanks for noticing, over time I (and Vik!) have honed the art of doing exactly what you say. As the Interactive Health consultant said when she called to complain about being outed in the Wall Street Journal and explained that the cost reduction [on page 8 of her report] and risk reduction [on page 9] had nothing to do with each other (???), and I said, “in other words, page 9 just coincidentally followed page 8.”

She said, “Al, I can’t talk to you if all you’re going to do is use my own words against me.”

Guest
Jul 11, 2013

This all does seem truly amazing – In the interest of fairness I have sent the initial blog to Dr. Paul Terry – the chief science officer of Staywell and have asked for his thoughts – – Jon

Guest
Jul 11, 2013

I contacted them too — the study’s author.

But you raise a good point about science– I am not challenging their “science.” I applied simple arithmetic to their own science. Here’s the difference: , measuring 5.6% risk reduction is “science”. I’m sure the science is wrong because 5.6% doesn’t take into account dropouts and non-participants. But nonetheless I accepted it. Then I applied arithmetic to the science and got the $17,000 figure.

They can only dig themselves in deeper by responding. For instance they might try to claim credit for people’s risks not going up. I hope they see this and change their mind (and perhaps contact me privately) because it will not be pretty if they try it publicly.

But I think the Staywell people are much smarter than the Interactive Health people, and will wait for this to fade away. BP won’t see this smackdown and that’s what counts. If they stir up the pot it might get back to BP.

Guest
Jul 12, 2013

I am happy to question the industry’s science. The notion that wellness vendors can diagnose anyone with anything, based on inputs to HRAs and annual biometric screens is not only not scientific, it’s ignorant and irresponsible.

While credible physicians and scientists labor over vital issues, such as does tight control of blood glucose improve outcomes and what is the best approach to determining whether an adult has hypertension (yes, it’s more complex than most people think), the confederacy of dunces populating wellness companies from coast to coast tells us that, presto, they can tell you in a snap whether or not you are diabetic or hypertensive.

Over the past decade, I have had three false positive readings of cardiometabolic markers. In the common wellness company scenario, I would have been encouraged to: accept my “diagnosis” and get about the business of being “treated.” Luckily for me, in all three cases, I was working for myself and with really smart docs, who said “let’s just sit on this reading because it doesn’t make any sense at all in your context. We’ll recheck things in a few months.” For all three markers, I never again had a positive finding, and never required treatment, and, in fact, have not had anyone of them remeasured any more frequently than the NHBLI timetable of every five years.

Corporate wellness? Science? It’s injurious to the concept of science that those words are even juxtaposed.

Guest
Mitch
Jul 12, 2013

Al and Vic are doing a great service with these articles, and the wellness industry needs to be called on the fallacy they peddle. As Al often points out creating a culture of wellness is hard, complex, takes a long time, and probably does not require a wellness program. The more serious issue he exposes (besides make up numbers, lies and incompetence) is that having a wellness program is kind of like taking an anti-depressant when your life is a mess. It obscures the real issues and deflects focus away from the hard work needed. HR can say “We did do something! We spent a lot on wellness! Look at all the pedometers are pretty water bottles!”

Guest
Jul 12, 2013

Mitch, you are right on with your assessment of the wellness industry’s propensity to put lipstick on pigs, so to speak. The conventional wellness approach encourages employers to distribute t-shirts and other logo-emblazoned chotchkes as core pieces of “motivating” employees. I have found that all it does is encourage people to ask for more chotchkes, while they move into the cafeteria where they’ll be fed food that’s intended to be profitable and efficient, not healthy and instructive.

One very large employer I know of distributed mini white boards (with the wellness program logo, of course) that leadership wanted taped to everyone’s office door with their health goals written on them. People hated them and hated the idea that they had to share their health aspirations/problems with the whole world. Of course, in the C-suite, you could not find a single one of those infernal boards.

Guest
Jul 17, 2013

I agree with a lot of the other comments. This was a great post.
Wellness Program –A way of living that encourages good physical and mental health. It is a balanced lifestyle that includes an emphasis on the body, mind and spirit.

Health Wellness Center

Guest
John
Jul 19, 2013

Even if these articles have valid points for debate, you might consider that these companies don’t want to engage Al Lewis because he has a propensity for accusations that make him look like an internet troll. Nobody wants to be drawn in to a flame-war manufactured to help somebody sell books at their own expense.

Guest
EtAli.Jr
Jan 12, 2015

Writing style aside, Mr. Lewis et. al. provide a very valuable service. The huge number of companies that have implemented such programs over the years and the costs of such programs begs the question (repeatedly) “where’s the beef”? I’ll settle for tofu or some veganoid substitute.

It’s just not there. Yet far too many have jumped on the wellness band wagon as it has far surprassed cotttage industry status and now looms large on the corporate landscape. Perhaps the C-suite does it for “corporate social responsibility” reasons? Perhaps it’s a distraction to placate associates with something that looks like it is an act done for them while the real work of hacking at insurance plan design goes on behind closed doors (anyone notice any excessive cost shifting of late?).

The numbers generated by biometry are an awkard part of this never ending drama. Again, what are they for? When a vendor draws one’s blood and peforms 30+ tests one has to realize something is way out of line. Such testing behavior raises many questions including what the devil are you going to do with all that useless if not harmful data (false positives)?

Yes, Al and company are somewhat taken with themselves; academic pedigrees seem quite important to them don’t they? Yet that posh academic experience they claim coupled with undoubtedly superior genetic substrate(s) has indeed produced the well oiled machine you witness when you read their blogs as well as the not so subtle microscopy they perform on errant thoughts and missives.

Style points are everything (an old girl friend once told me). I say there is a lot of room for different styles. Don’t you?

Guest
Al Lewis
Jan 13, 2015

Thanks for your comment and keep ’em coming.

Your theory may very well be accurate (I hadn’t really thought of it before) about why no one defends themselves. Others might not want to “get into the gutter.”

However, that doesn’t explain why, 24 named-vendor smackdowns later, none of the perps has even sent us a cease-and-desist letter, let alone sued us. We’re doing the best we can to encourage it, but so far no takers. We could use the PR and we would destroy them. We only publish a fraction of what we know because we can use only publicly available stuff. For many vendors we have ex-employee sources that usually say things like: “It’s even worse than you think.”

Al

PS If you don’t like the writing style on this blog, you may detest our book Surviving Workplace Wellness. Or you may not. Hmm… The only way to know is, Why don’t you buy one and decide for yourself? http://www.amazon.com/Surviving-Workplace-Wellness-Dignity-Finances-ebook/dp/B00HZR2EBE/ref=sr_1_1?ie=UTF8&qid=1390488623&sr=8-1&keywords=surviving+workplace+wellness

Guest
Jul 19, 2013

thank you for your boldly anonymous comment. One of the interesting things about the wellness ignorati is that they don’t understand the way math works. Math is not a “debate” with points and counterpoints. Staywell claimed 100x the savings for BP it claims on its own website. It’s their own math — both statements can’t be true. Perhaps they should “debate” themselves.

Latest http://jobs.aol.com/articles/2013/07/19/employer-wellness-programs-bad-health/

Guest
John
Jul 19, 2013

Why, were you interested in ad-hominem as a mathematical proof? My comment is on your rhetorical style, and it’s just a suggestion. Math equations never result in statements like “Along with their distaste for facts is a distaste for arithmetic.”

Guest
Michael
Jul 22, 2013

I read the BP article before being pointed to this post and I’m trying to follow your argument. I don’t quite understand how you arrived at your $17,000/member savings. You multiply the number of participants (20,343) by 5.6%, which you say is the number of people who experienced a reduction in risk. The article says, however, that 5.6% is the reduction in “health risks” among the population. I took that to mean, for example, that the baseline average for a health risk score was 100 and the the sample year average was 94.4, a 5.6% reduction in risk and not 5.6% of the population reduced risk. Can you clarify?

Guest
Jul 22, 2013

Good question–thanks for the opportunity to clarify and you have an interesting point, though not one that changes the answer much.

The article says 5.6% is the reduction in the “average number of lifestyle-related risks” among “repeat participants.” That means if there were 20000 repeat participants, about 1100 might have temporarily (for that year between readings) lost some weight or quit smoking.

There are a huge humber of things wrong with the science here. To being with, most of those 1100, statistically speaking, will regain the weight and/or start smoking again. And also the concept of limiting your count of the risk factors to people who were motivated enough to participate twice is ludicrous: it leaves out the dropouts and non-participants who are much more likely to see a risk increase.

My result is simply taking the total savings and dividing it by those 1100 people whose risk declined.

However, if you assume THREE risk factors per person (and forgetting still about not counting the dropouts), then the total number of risk factors that went down would be 5.6% of 20,000 * 3, or roughly 3360 risk factors.

Now, divide that $20MM in savings by 3360 risk factors and you get about $6000/risk factor. This is a bit more than total spending per person, so it’s still impossible, and it’s still 60 times what Staywell says is possible. (Staywell is also overestimating what is possible on its own website, which we’ll address in another posting soon.)