British Petroleum’s Wellness Program is Spewing Invalidity

A critical observation in Cracking Health Costs is you need not “challenge the data” to invalidate claims that wellness saves money.  Instead, you can simply read the data as presented.  You’ll find it usually invalidates itself.

Nowhere is that more true than in a study published this month by Mercer, Staywell and British Petroleum (“BP America”) in the Journal of Occupational and Environmental Medicine (JOEM).   As we’ll demonstrate, the results completely contradict Staywell’s own statements, and are also mathematically impossible.  Indeed, Mercer was a wise partner choice by BP America because their validations are often unconstrained by the limits of possibility.   For instance, they validated massive savings both for infants in a North Carolina Medicaid program that did not enroll infants, and for a Georgia Medicaid disease management program that did not manage diseases, at least according to the FBI.

Along those lines, let’s see what happens when one compares the JOEM conclusion — that the Staywell wellness program for BP America achieved almost $20,000,000 in savings on 20,343 BP participants after only two short years – to the limits of possibility.

It turns out this overall savings claim of $1,000/person would require completely wiping out wellness-sensitive medical events (heart attacks, diabetes events etc.) not just on those 20,000+ people, but also on perhaps 40,000 of their closest friends.  The authors elected not to disclose the change in wellness-sensitive medical events across the entire eligible population, perhaps because they were embarrassed by the size of the decline, if indeed those events declined at all.

The authors went a step farther and decided not to even acknowledge that a wellness program should reduce wellness-sensitive medical events, a perfectly logical nexus not unlike saying that an antihyperlipidemic drug should reduce cholesterol levels or an asthma rescue drug should improve breathing, or sunblock should prevent sunburn.

It’s not just the two of us who think logic should prevail in connecting wellness initiatives to wellness events.   Health Affairs laid them out too, in a seminal 2013 study that Mercer and Staywell didn’t footnote despite being the most recent scholarly article on exactly this same topic.  They also selectively omitted other key citations.  The equally seminal RAND study – which concludes that wellness programs produce no two-year population-wide savings – also went unacknowledged.

Curiously biometric screens were noted as one of the sources of BP’s alleged savings, but this once again turns out to breach the wall of possibility:   another – also unacknowledged — Health Affairs article showed that it is mathematically impossible not to lose money on biometric screens, which is why government agencies, expert panels and books warn against using them.  (They can and do also harm patients, which is perhaps why most people only participate when threatened with large penalties, as was the case with BP, where 30% of their employees elected to forfeit $1200, to stay out of their program.)

These selective citation omissions are indicative of the wellness industry researchers’ distaste for facts.  This is not to say that wellness researchers are stupid.  Quite the opposite: they are smart enough to realize that facts are their worst nightmare.

The Data that Didn’t Bark in the Nighttime

And in this case, the most troubling fact is right in their own study, albeit buried deep enough (page eight) that the JOEM peer reviewers may not have noticed it.  That is the percentage of people whose risk factors actually improved:  5.6%.  As modest as this figure is, it actually excludes the people most likely to get worse– those who dropped out of the program and those who wanted nothing to do with it in the first place.

Understandably, the study authors decided not to draw attention to the simple division of the savings ($19,700,000) into the number of people who even theoretically could have avoided wellness-sensitive medical events (5.6% of 20,343, or 1,139 people), because that division would have yielded a whopping $17,300 in savings per person whose risks declined.

And once, again, the limits of possibility don’t stand a chance against Mercer and Staywell, for three reasons.  $17,300 is:

  1. Almost three times what a company spends in total on the average covered person`.

  2. At least 30 times what a company spends on wellness-sensitive medical events for the average covered person.

  3. More than 100 times what the lead Staywell author says on Staywell’s own website is possible.

This assumes there were any savings at all, which we’ll never know because recall they failed to disclose the wellness-sensitive medical event tally.

Lessons for Wellness Researchers

So, what can be learned from this valuable addition to our growing library of teaching tools for population health management, other than how easy it is for vendors and consultants to convince their HR clients in large corporations that they are saving massive sums of money through wellness?

First, the most sophisticated-sounding study designs in the world – and this one looks at least six figures worth of sophistication – can’t ignore the question:  “Is this result even mathematically possible?”

Second, those expensive “matched control” study designs are worthless anyway when comparing actively motivated participants to unmotivated non-participants.  You can match demographics but you can’t match the difference in mindset between (for example) smokers who want to quit and smokers who don’t.  Health Fitness Corporation showed that would-be participants significantly outperform non-participants even without a program.

Third, researchers need to stop attributing all good things to wellness and instead employ a basic biostatistical tenet, which is that the outcome should bear some relationship to the intervention.  In the case of wellness, that means a program designed to reduce wellness-sensitive risks should have its greatest if not only effect in avoidance of medical events of a wellness-sensitive nature.

Fourth, they need to understand the relationship between risk factor reduction and cost savings.  As Why Nobody Believes the Numbers shows mathematically, costs decline about a tenth as much as risk factors decline, meaning that BP could attribute roughly a 0.56% decline in costs to the 5.6% risk factor decline.  This 0.56% gross decline would be offset by more preventive physician visits, more preventive drugs, the cost of the program and incentives.  It also assumes that the 5.6% itself figure is real, which it almost certainly isn’t because…

…Fifth, this industry needs to stop ignoring the overwhelming preponderance of facts that don’t support its belief system.  In particular, it needs to stop citing the single company (Johnson & Johnson) which has a strong wellness culture that few others have duplicated , and instead look at RAND and (for example) IBM, Dow Chemical or Salt Lake County to see a much more realistic view of what companies can accomplish.   Wellness industry leaders specialize in distorting fundamental scientific and analytic principles to meet their own needs.  Increasingly, however, health policy experts are calling them on this, which means that there is hope for introducing integrity into the wellness outcomes reporting process.

Along with their distaste for facts is a distaste for arithmetic.  Page eight of the JOEM paper asserts that “there are no industry standards on how to calculate ROI…”  They could easily have added:  “…because most vendors, including us, refuse to disclose the rate of wellness-sensitive medical events, which would be the obvious choice as an industry standard if we were able to actually reduce these events.” Instead the vendor association uses an invalid consensus methodology, making wellness/disease management the only industry where mathematical proof is trumped by popular vote.

Finally, industry supporters need to stop citing the single article in a first-tier journal with a positive finding about wellness’s impact on health spending.  That article is now approaching its fourth birthday, it was a meta-analysis of studies mostly drawn from third-tier journals that among them had never published an article that found anything other than positive results from wellness (there is even a term for this — “publication bias”), and even its authors no longer seriously defend it.  (Reflecting the dearth of more recent positive press in first-tier journals, a leading wellness proponent recently wrote that this article “should be cited much more frequently by health promotion and wellness professionals.)

This industry needs to acknowledge that almost all credible recent literature – ours or anyone else’s –finds largely failure, and then develop or identify some best practices.   Hence we once again invite people to identify a single wellness vendor that saves money by reducing population-wide wellness-sensitive medical events, for us to highlight in a future column.

Al Lewis is the author of Why Nobody Believes the Numbers, co-author of Cracking Health Costs: How to Cut Your Company’s Health Costs and Provide Employees Better Care, and president of the Disease Management Purchasing Consortium.

Vik Khanna is a St. Louis-based independent health consultant with extensive experience in managed care and wellness.  An iconoclast to the core, he is the author of the Khanna On Health Blog.  He is also the Wellness Editor-At-Large for THCB.

Categories: Uncategorized

Tagged as: , , ,

28 replies »

  1. By the way, fast-forward to the smashing conclusion: after reading this expose, the wellness ignorati on the Koop Committee got together and decided that the best response to having two of their sponsors (Mercer and Staywell) exposed for their fraudulent outcomes is: Let’s give them an award. http://theysaidwhat.net/2015/01/05/goetzel-koop-committee-staywell-mercer-bp-america-meet-groundhog-day/

    Then Ron Goetzel’s (who runs the committee that gives out the award) says we are bullying him for asking him questions like, why he always gives awards to his sponsors no matter how obviously corrupt and dishonest their programs are. http://theysaidwhat.net/2015/02/28/goetzel-apology-2/

  2. Thanks for your comment and keep ’em coming.

    Your theory may very well be accurate (I hadn’t really thought of it before) about why no one defends themselves. Others might not want to “get into the gutter.”

    However, that doesn’t explain why, 24 named-vendor smackdowns later, none of the perps has even sent us a cease-and-desist letter, let alone sued us. We’re doing the best we can to encourage it, but so far no takers. We could use the PR and we would destroy them. We only publish a fraction of what we know because we can use only publicly available stuff. For many vendors we have ex-employee sources that usually say things like: “It’s even worse than you think.”


    PS If you don’t like the writing style on this blog, you may detest our book Surviving Workplace Wellness. Or you may not. Hmm… The only way to know is, Why don’t you buy one and decide for yourself? http://www.amazon.com/Surviving-Workplace-Wellness-Dignity-Finances-ebook/dp/B00HZR2EBE/ref=sr_1_1?ie=UTF8&qid=1390488623&sr=8-1&keywords=surviving+workplace+wellness

  3. The Wellness Industry, where big bugs are involved, has left the Healthcare arena into layman’s arena. Everyone who can use their ‘Google finger’ to look up any subject believe they are now experts. Unfortunately they do not understand the complexities of interpreting the data.
    Physicians who deliver healthcare and manage risk factors in their patients and Medical epidemiologists should be vetting these so called Wellness vendors. Unfortunately the appropriate credentialed professionals are no longer in charge

  4. Writing style aside, Mr. Lewis et. al. provide a very valuable service. The huge number of companies that have implemented such programs over the years and the costs of such programs begs the question (repeatedly) “where’s the beef”? I’ll settle for tofu or some veganoid substitute.

    It’s just not there. Yet far too many have jumped on the wellness band wagon as it has far surprassed cotttage industry status and now looms large on the corporate landscape. Perhaps the C-suite does it for “corporate social responsibility” reasons? Perhaps it’s a distraction to placate associates with something that looks like it is an act done for them while the real work of hacking at insurance plan design goes on behind closed doors (anyone notice any excessive cost shifting of late?).

    The numbers generated by biometry are an awkard part of this never ending drama. Again, what are they for? When a vendor draws one’s blood and peforms 30+ tests one has to realize something is way out of line. Such testing behavior raises many questions including what the devil are you going to do with all that useless if not harmful data (false positives)?

    Yes, Al and company are somewhat taken with themselves; academic pedigrees seem quite important to them don’t they? Yet that posh academic experience they claim coupled with undoubtedly superior genetic substrate(s) has indeed produced the well oiled machine you witness when you read their blogs as well as the not so subtle microscopy they perform on errant thoughts and missives.

    Style points are everything (an old girl friend once told me). I say there is a lot of room for different styles. Don’t you?

  5. What they don’t tell you is if you don’t participate they double or triple your health insurance premiums.

  6. The research shows that the biggest impact of health behavior comes from the patient’s encounter with their physician. A one on one encounter. Individuals are more likely to take the advise of a physician and follow through on health behavior than based on some Wellness Program.

  7. sorry–1139. Was in a hurry and miswrote from memory as opposed to looking back. Number of people who perservered enough to be measured twice, and showed success

  8. I realize I don’t have much of a chance of convincing you, since as Upton Sinclair noted, it’s impossible to prove something to someone whose salary depends on believing hte opposite, but here goes:

    (1) like it or not, only 1103 people’s risk factors declined. Sorry. Dropouts and non-participants were much more likely to have risk factor increases, os probably on balance there was close to no change.

    (2) frankly it wouldn’t matter because BP’s 20,000 employees don’t even incur $20MM in wellness-sensitive medical events to avoid. (You are confusing total spending on chronic disease wth reducible events.)

    (3) Since you don’t understand the concept of wellness-sensitive medical events, you should probably learn what they are before getting in over your head here.

    (4) It probably is the case that healthier people do better in surgeries but wellness programs that only make 1103 people a little healthier aren’t going to make a bit of difference.

    (5) You contradicted yourself. BP says it saved $20MM by basically managing current cases (in a 2-year period) so whatever elusive long-term goal you are referring to, even if it could be achieved, wouldn’t hvae counted towards that $20-MM.

    By the way, the wheels have completely fallen off your wellness bus. Even Kate Baicker — whose article provides the only ROI support for wellness in a real journal (not the JOEM etc., which are house organs) — just recanted. You are now left with no articles in any real journal on your side, with more bad news coming out in a real journal within a few months.

    And your ethically challenged C Everett Koop Award Committee won’t retract the award it gave to Nebraska even though Nebraska has admitted fabricating the data. What would Dr Koop think of that?

    So once you have an industry that has any semblance of intellectual backing and ethics, you can raise more points. But you people better get your own house in order first.

  9. Al, as requested, here are bullets from our LinkedIn discussion regarding the flaws in your analysis to allow for a point counterpoint discussion.
    1. Your analysis hinges on the 5.6%, which you refer to as percentage of people whose risk factors actually improved, but the 5.6% is actually the reduction in the average number of lifestyle-related health risks among all repeat HQ participants. So you are taking an average reduction across the entire population and using it as if it were a percentage of the population that were impacted to come up with your unbelievable cost reduction figure. Since your whole article hinges on this one factor, the rest of your conclusions seem to fall apart.
    2. You rely an awful lot upon a $400 figure for a theoretical maximum, which seems too low. From the Medical Expenditure Panel Surveys average of 2006 through 2008 the costs associated with risk factor-related conditions account for 36% of total medical expenditures, or approximately $1,285 per working age adult (presumably the costs and potential impact has increased since then). Recognizing that not all of the costs are associated with lifestyle factors, epidemiological studies through the Global Burden of Disease study developed attribution factors, which when applied to these conditions, reduces this figure to approximately $649 per working age adult. Some fraction of this can be influenced by wellness if you were able to get everyone to their optimal health, but this is still more than your theoretical maximum of $400.

    In addition to looking at the current costs of those with some risk factor related health condition, there are aspects of wellness that don’t get accounted for in looking at current cost levels related to risk factor-related conditions:

    1) Impact wellness has on care outcomes – We know that if someone is physically, emotionally and spiritually healthier, their procedure success rate, recovery time, ALOS and chance of no complication/readmission is better, pretty much for any type of procedure.
    2) Reduction of future incidence – If someone is currently healthy, but not exhibiting healthy lifestyle factors, there is a higher incidence that they will contract a chronic condition in the future.
    3) Management of Incidence – If the current unhealthy behaviors continue, the current costs will increase in the future. If people can change their behaviors, presumably it would influence by the behaviors of those with the conditions. This is a hard one to prove, so I say presumably, but ultimately the goal of wellness is to reduce or eliminate the development of future cases, not manage current ones.

    3. You point to the Rand study as some gospel, but the four programs they evaluated are clearly ineffective programs based on the fact that they averaged only 40% participation. I wouldn’t point to that as a good analysis of effective programs. There are effective and ineffective programs and we as a profession need to be clear about the distinctions and what makes the difference.

  10. Good question–thanks for the opportunity to clarify and you have an interesting point, though not one that changes the answer much.

    The article says 5.6% is the reduction in the “average number of lifestyle-related risks” among “repeat participants.” That means if there were 20000 repeat participants, about 1100 might have temporarily (for that year between readings) lost some weight or quit smoking.

    There are a huge humber of things wrong with the science here. To being with, most of those 1100, statistically speaking, will regain the weight and/or start smoking again. And also the concept of limiting your count of the risk factors to people who were motivated enough to participate twice is ludicrous: it leaves out the dropouts and non-participants who are much more likely to see a risk increase.

    My result is simply taking the total savings and dividing it by those 1100 people whose risk declined.

    However, if you assume THREE risk factors per person (and forgetting still about not counting the dropouts), then the total number of risk factors that went down would be 5.6% of 20,000 * 3, or roughly 3360 risk factors.

    Now, divide that $20MM in savings by 3360 risk factors and you get about $6000/risk factor. This is a bit more than total spending per person, so it’s still impossible, and it’s still 60 times what Staywell says is possible. (Staywell is also overestimating what is possible on its own website, which we’ll address in another posting soon.)

  11. I read the BP article before being pointed to this post and I’m trying to follow your argument. I don’t quite understand how you arrived at your $17,000/member savings. You multiply the number of participants (20,343) by 5.6%, which you say is the number of people who experienced a reduction in risk. The article says, however, that 5.6% is the reduction in “health risks” among the population. I took that to mean, for example, that the baseline average for a health risk score was 100 and the the sample year average was 94.4, a 5.6% reduction in risk and not 5.6% of the population reduced risk. Can you clarify?

  12. Why, were you interested in ad-hominem as a mathematical proof? My comment is on your rhetorical style, and it’s just a suggestion. Math equations never result in statements like “Along with their distaste for facts is a distaste for arithmetic.”

  13. thank you for your boldly anonymous comment. One of the interesting things about the wellness ignorati is that they don’t understand the way math works. Math is not a “debate” with points and counterpoints. Staywell claimed 100x the savings for BP it claims on its own website. It’s their own math — both statements can’t be true. Perhaps they should “debate” themselves.

    Latest http://jobs.aol.com/articles/2013/07/19/employer-wellness-programs-bad-health/

  14. Even if these articles have valid points for debate, you might consider that these companies don’t want to engage Al Lewis because he has a propensity for accusations that make him look like an internet troll. Nobody wants to be drawn in to a flame-war manufactured to help somebody sell books at their own expense.

  15. I am happy to question the industry’s science. The notion that wellness vendors can diagnose anyone with anything, based on inputs to HRAs and annual biometric screens is not only not scientific, it’s ignorant and irresponsible.

    While credible physicians and scientists labor over vital issues, such as does tight control of blood glucose improve outcomes and what is the best approach to determining whether an adult has hypertension (yes, it’s more complex than most people think), the confederacy of dunces populating wellness companies from coast to coast tells us that, presto, they can tell you in a snap whether or not you are diabetic or hypertensive.

    Over the past decade, I have had three false positive readings of cardiometabolic markers. In the common wellness company scenario, I would have been encouraged to: accept my “diagnosis” and get about the business of being “treated.” Luckily for me, in all three cases, I was working for myself and with really smart docs, who said “let’s just sit on this reading because it doesn’t make any sense at all in your context. We’ll recheck things in a few months.” For all three markers, I never again had a positive finding, and never required treatment, and, in fact, have not had anyone of them remeasured any more frequently than the NHBLI timetable of every five years.

    Corporate wellness? Science? It’s injurious to the concept of science that those words are even juxtaposed.

  16. Mitch, you are right on with your assessment of the wellness industry’s propensity to put lipstick on pigs, so to speak. The conventional wellness approach encourages employers to distribute t-shirts and other logo-emblazoned chotchkes as core pieces of “motivating” employees. I have found that all it does is encourage people to ask for more chotchkes, while they move into the cafeteria where they’ll be fed food that’s intended to be profitable and efficient, not healthy and instructive.

    One very large employer I know of distributed mini white boards (with the wellness program logo, of course) that leadership wanted taped to everyone’s office door with their health goals written on them. People hated them and hated the idea that they had to share their health aspirations/problems with the whole world. Of course, in the C-suite, you could not find a single one of those infernal boards.

  17. Al and Vic are doing a great service with these articles, and the wellness industry needs to be called on the fallacy they peddle. As Al often points out creating a culture of wellness is hard, complex, takes a long time, and probably does not require a wellness program. The more serious issue he exposes (besides make up numbers, lies and incompetence) is that having a wellness program is kind of like taking an anti-depressant when your life is a mess. It obscures the real issues and deflects focus away from the hard work needed. HR can say “We did do something! We spent a lot on wellness! Look at all the pedometers are pretty water bottles!”

  18. I contacted them too — the study’s author.

    But you raise a good point about science– I am not challenging their “science.” I applied simple arithmetic to their own science. Here’s the difference: , measuring 5.6% risk reduction is “science”. I’m sure the science is wrong because 5.6% doesn’t take into account dropouts and non-participants. But nonetheless I accepted it. Then I applied arithmetic to the science and got the $17,000 figure.

    They can only dig themselves in deeper by responding. For instance they might try to claim credit for people’s risks not going up. I hope they see this and change their mind (and perhaps contact me privately) because it will not be pretty if they try it publicly.

    But I think the Staywell people are much smarter than the Interactive Health people, and will wait for this to fade away. BP won’t see this smackdown and that’s what counts. If they stir up the pot it might get back to BP.

  19. This all does seem truly amazing – In the interest of fairness I have sent the initial blog to Dr. Paul Terry – the chief science officer of Staywell and have asked for his thoughts – – Jon

  20. Yes, like you and like Diogenes, we are also looking for an honest story of savings. It’s a good question and you are not the first one to ask it.

    Specifcally, I don’t want to imply that all vendors and consultants make up outcomes, not by any stretch. 80% are honest…and don’t release their outcomes because they know they don’t save money. There are many reasons to have a culture of wellness as an end in itself, but reducing healthcare spending is not one of them.

    And thanks for noticing, over time I (and Vik!) have honed the art of doing exactly what you say. As the Interactive Health consultant said when she called to complain about being outed in the Wall Street Journal and explained that the cost reduction [on page 8 of her report] and risk reduction [on page 9] had nothing to do with each other (???), and I said, “in other words, page 9 just coincidentally followed page 8.”

    She said, “Al, I can’t talk to you if all you’re going to do is use my own words against me.”

  21. Agree with Mary – the vendors are not disposed to let their clients know the data has issues…too much $$$ at stake. Responsible consultants own up to their mistakes and fix their methods.

  22. These stories of yours are always hilarious, how you call them idiots without calling them idiots just by quoting their own data. What does BP think of this? How about Mercer or the vendor? Isn’t there anyone who is doing this right? I’d love to read a good-news story even if the bad news ones make for better copy. Al is the definitive “go to” source to spot the insanity and just plain idiocy of claims a described above.

  23. You apparently haven’t read Al’s book, which is surprising because I’ve seen your comments on his postings before. He says the wellness industry never resonds or even acknowledges his things, in order not to draw attention to them. Doubt stay well and mercer have even told BP

  24. Sam: we, too, would like to read rebuttals. We relish the rhetorical brawl. But, regardless of whether we write for THCB, Health Affairs, the Wall Street Journal, (or other pending outlets), the industry is silent, save for the few wellness acolytes who call us names on LinkedIn. Debate is a competitive process, and when the opposition not only refuses to show up and engage, but they won’t even defend the claims they’ve already made in supposedly reputable peer-reviewed journals, it tells you an lot about both their courage and their integrity.

    Al and I are not done, not as long as they keep claiming impossible savings and improvements in health and using those same false claims to dupe (sometimes unwitting, but sometimes complicit) employers and to ramrod employees into things that they don’t want to do (such as take HRAs and submit to biometric screenings). We aren’t done by a long shot.

  25. When I read these essays on wellness, they all seem so cut-and-dried — I’d like to see a rebuttal by the named parties

  26. Excellent article. I have been saying the same thing for years without any statistical proof, just common sense and logic. I once managed a wellness program and spent over $2 million a year doing it. Nobody asked about hard savings or proof of savings (there weren’t any), but nevertheless it was instinctively and politically a good thing to do so the money was spent year after year and still is to this day. Employees think they save on their premium by participating. In fact, the supposed premium savings are fictitious because the discount they receive was added to the premium. Wellness in the corporate setting is a house of cards that nobody has the guts to blow at.