When Foundation-Approved Apps Founder

What does it mean when an app wins a major foundation’s developer challenge, and then isn’t updated for two and a half years?

Today, as I was doing a little background research on task management apps for caregivers, I came across a 2012 post listing Pain Care as a handy app for caregivers.

Pain is certainly something that comes up a lot when it comes to geriatrics and supporting caregivers, so I decided to learn a little more about this app.

“The Pain Care app won the “Project HealthDesign” challenge by the Robert Wood Johnson Foundation and California HealthCare Foundation,” reads the descriptive text in the Google Play Store.

Well, well, well! RWJF and CHCF are big respectable players in my world, so I was impressed.

But then as I looked at the user reviews, I noticed something odd. Namely, that the most recent one seems to be from April 2012, which is like 2-3 generations ago when it comes to apps.

And furthermore, the app itself was last updated in February 2011. This is like a lifetime ago when it comes to apps.

I decided to download the app and give it a whirl. It’s ok. Seems to be an app for journaling and documenting pain episodes, along with associated triggers. Really looks like something developed by doctors: one of the options for describing the type of pain is “lancinating,” and in a list of “side-effects” (side effects of what? the pain medication one may have just taken?) there is the option to check “sexual dysfunction.” Or you could check “Difficulty with breathing.” (In case you just overdosed on your opiates, perhaps.)

The app does connect to a browser-based account where I was able to view a summary of the pain episode I’d documented. It looked like something that one should print and give to a doctor, and in truth, it would probably be helpful.

Setting snarky comments about the vocabulary aside: this app actually looks like a good start for a pain journal. But it needs improvement and refining, in order to improve usability and quality. Also, although I don’t know much about app development and maintenance, I assume that apps should be periodically upgraded to maintain good performance as the operating systems of iPhones and Android phones evolve.

What does it mean, that this app was blessed by RWJF but then has been left to founder? A quick look at the developer’s news feed reveals that the app maker, Ringful Health, has racked up an impressive array of research contracts and prestigious partnerships: NIHCMSConsumer Reports.

Will these alliances lead to more lasting (read: supported and improved in an ongoing fashion) products and apps that can benefit patients and caregivers? What is the measure of a successful app, from the perspective of public health authorities, and of foundations?

Clearly, this is partly about issues related to business case and funding. To maintain an app, you need money. (If you get money from active users, then you definitely need to work on keeping them happy.)

Who will pay to maintain the apps that foundations, government agencies, and public-interest agencies help start?

And what does happen to most of these apps and tech projects that win foundation awards? Would be interesting to learn more about the natural history of such apps…

Leslie Kernisan, MD, MPH, has been practicing geriatrics since 2006, and is board-certified in Internal Medicine and in Geriatric Medicine. She is a regular THCB contributor, and blogs at GeriTech.

11 replies »

  1. Michael Yuan, thank you so much for sharing your perspective as a developer.

    Rebecca, I’m glad you were able to clarify some of ONCs thinking in sponsoring these challenges.

    I have to say, if misconceptions about the purpose of these challenges are common, then perhaps some changes in marketing and messaging might be eventually be helpful…

  2. I thought I would chime in since the Division I work in at ONC sponsors many Challenges, including many that have been run through Health 2.0.

    A common misconception is that the primary intended outcome of Challenges is for the winning application to turn directly into a successful business. This perspective is usually held by people who have never actually worked in a startup or brought software to market. Anybody who has knows that a few weeks in you are just barely beginning to understand the problem and what you should build. At least for a startup, they have the incentive to keep iterating on the problem until they have a solid business case and growing user base.

    I respectfully suggest that this is the biggest barrier to most of the mobile apps and other products coming out of academia. Unless the creator is willing to step away from clinical practice and actually build a viable company, which is the HARD PART, that app continuing to improve is out of the goodness of somebody’s heart or inherently unsustainable research and grant money.

    A few but not all of the many reasons we sponsor Challenges:
    1. To attract talented entrepreneurs and developers to the healthcare field.
    2. To bring their attention to a high priority patient population or other software/app use case.
    3. To get devs playing with new data or important standards for data transport that they may not be aware of, especially if not coming from a health IT/informatics background.
    4. To create a structured experience that encourages developers, clinicians, and patients to work together. See the ONC’s recent Patient CoDesign Challenge.
    5. To ask some of the best minds out there to advance our thinking around a specific aspect of the technology. For example, the Blue Button Design Challenge held last fall attracted some of the best designers in the world to reimagine how we represent health data to patients. The results were never intended to be a product, but they were stunning and greatly advanced the perception of what’s possible using an otherwise difficult to interpret data document structure.

  3. We are the app developer. We are also disappointed with the outcome of the app. But I think we also learned valuable lessons here.

    One of the challenges small business facing is the need to rapid prototype and test the market, and then move on to another idea when the previous idea fails to gain traction. That is especially true with grant funded projects — they need to “make money” after the grant ends in order to justify continued development effort.

    Pain Care was developed in the early days of mHealth, and it was indeed very physician focused — the reason is that we believe we must engage physicians to look at the data. We still hold that belief. It is a learning process for us. We put in our own money to develop the app, and fortunately, won the developer challenge.

    We made the app public after the challenge to “test the market” — so to speak. But, as you know, essentially *none* of the pure app-based “patient journal” has turned out to be a success (let alone a financial success). Our app is no exception. It is enormously costly keep the app updated for all those iPhone, iPad, iOS released every year, as well as thousands of Android devices released since then.

    So, the app becomes one of those “outdated” apps in the app store, and I think it is quite obvious to users as well. However, I think the app did contribute significantly to the “science” of mHealth. We now understand much more what works and what not in “patient engagement”. Many other “pain management” apps have since emerged, and many have done a better job than ours. I think that was what RWJF wanted when they challenged developers back then. 🙂

    Today, we do things a lot differently. We no longer release research grant-funded apps to the public. Instead, we run clinical studies to test them in much smaller / controlled groups. We do not attempt to tackle vague “big problems” like general pain management any more — instead, we are much more focused on managing specific diseases that include pain. We are also moving beyond “pure software” and “simple reminders” to engage people in multiple modalities.

    All of these would not be possible without the generous award RWJF gave us in picking Pain Care as the winner of one of the first developer challenges.

    Michael Yuan, Ringful Health

  4. Thanks all for these comments!

    @ Mike Painter, thank you so much for taking the time to share the RWJF perspective. I had indeed figured that this was early on in everyone’s experience with apps and challenges; sorry if I didn’t make that clear in the post.

    In general, I’ve often noticed that my colleagues in academia create high-quality tools for clinicians or patients or caregivers and then they are never widely disseminated, or improved upon.

    Whereas many from the business and tech community have lots of experience figuring out how to get products out there and used/bought by people.

    So, should be a good match if only we can figure out how to make the match work well…I’m very glad to see public-minded entities such as foundations and government agencies trying to foster these partnerships. Am still waiting to see a product of a challenge take off widely but expect to see it soon…

  5. RWJF here.

    Interesting post and conversation as always on THCB. The original post and comments all make some great points. For us at RWJF, though, there is important context regarding this particular Project HealthDesign challenge and app in question–perhaps most importantly: that challenge was a very early one for us–and arguably for the field in general. We were learning and continue to learn in fact about the role of challenges in general for us–and about challenges that promote app development in particular.

    We ran this challenge back in the summer 2010. It was among the first for us; the winner received a $2,500 prize. In grant terms for a foundation of our size, that amount is a very, very small award. Our purposes at the time for this particular app challenge were really two: 1) try out the app challenge idea; and 2) bring attention to the Project HealthDesign grantees’ design work. From our perspective, this small challenge succeeded on both counts–we learned about challenges as a funding tool. It also allowed us to bring attention to the project and the grantees in ways we would not have otherwise been able to do.

    At the time (2010), nobody really knew where apps were going in spite of the buzz and interest. We also certainly did not know where apps generated by this particular challenge might go–whether they might flourish or stall. We were interested to see what developers might come up with, but we also very specifically were not interested in a long term relationship with the winners. In fact the point of that challenge was to award a no-strings-attached prize. Our sense was that a given winning app would succeed or fail based on the merits of the app and the whims of the developing marketplace. We did not want to take on the role of plumping the winner-or for that matter helping develop a particular marketplace for such apps.

    We did learn a great deal about challenges even with this small, fledgling effort–and proceeded to conduct several more challenges–including two active Health 2.0 challenges. For instance, we’ll be announcing winners of our Aligning Forces Game challenge and the Data Visualization challenge at the Health 2.0 conference in a few weeks.

    In spite of that current RWJF challenge activity–and some increase in the size of subsequent prizes, this prize funding tool remains for us a tiny fraction of the amount of funding we provide by grant or program related investment, annually. Also as our challenges have gotten more complex with larger prizes, we continue to learn about their role–and how RWJF might or even should interact with winners after the challenge. That is all definitely a work in progress for us.

    We do, however, believe that there is an ongoing role for challenges–including perhaps challenges that promote app development.

    Thanks again for reminding us all about this early one.

  6. All these observations are right – when apps are thought of as “toys” and not useful tools for managing a problem or health generally, that’s what will happen. I’m hoping that different “apps” (I use this term loosely) will prove more effective – cost-wise and convenience-wise for patients – so that “less” of other stuff is needed. Medications, hospital and custodial care, face to face clinician visits, etc. That will take more well thought out stuff than what’s currently seen as “hot” today. The pain management app is an example of some I’ve seen tested and used for other single-purpose issues – RA, IBD, pediatric migraines, etc. The goal is to get it to seamlessly integrate into clinicians’ records so patients don’t have to print off a report and bring it to a face to face visit. If you’re having pain issues right now, will it help to give a report to your physician in a month? Marginally, maybe, but if there were ways to monitor data in real time, that data could be used to give patients real relief for ongoing issues and help them avoid more costly care. I think Open mHealth (Ida Sim, et al.) is trying to get there – that would be awesome if they could succeed.

    I imagine a day when each drug, device, etc., comes with a HIT component that works in the user-agnostic health data world (patient/provider access, of course) to facilitate its optimal use. Then drug companies wouldn’t have to create a website for each drug they make to help consumers utilize their product effectively.

  7. @ Peter Elias

    What does it mean?

    Well, to me this is about sustainable development. Too many people are doing work like this and moving on to the next product and the next contest rather than building products that are carefully thought out and designed to evolve and succeed. This is a fundamental philosophical choice.

    Why are people doing this? Certainly not because they’re stupid. They’re doing this because this is what the system incentivizes them to do so.

    I’d like to see a study that scientifically tracks the life cycle of products in the app economy. And oh – while we’re at it — let’s remember that this problem isn’t just limited to apps. The problem also extends to disposable business models …

  8. This is a free app? From what see it appears to be. There are two points here. One is that free apps need to have some revenue in order for it to be updated and promoted and kept alive in the eyes of the users and new users. It is becoming apparent that paying for an app has a huge benefit for the user re: support and updates. In app sales may help but it is hard to do with health related apps.

    The second point is that the app maker may have used this app to get press, get recommendations that were converted to contracts. Yes I am a cynic.

  9. What does it mean?

    Perhaps it means that the app was built and is owned/maintained by people whose goal was to create and sell something. They did so, accumulating some accolades in the process, and have moved on.