Should Wearables Data Live In Your Electronic Medical Record?

Shaywitz of EldredThe great promise of wearables for medicine includes the opportunity for health measurement to participate more naturally in the flow of our lives, and provide a richer and more nuanced assessment of phenotype than that offered by the traditional labs and blood pressure assessments now found in our medical record.  Health, as we appreciate, exists outside the four walls of a clinical or hospital, and wearables (as now championed by AppleGoogle, and others) would seem to offer an obvious vehicle to mediate our increasingly expansive perspective.

The big data vision here, of course, would be to develop an integrated database that includes genomic data, traditional EMR/clinical data, and wearable data, with the idea that these should provide the basis for more precise understanding of patients and disease, and provide more granular insight into effective interventions.  This has been one of the ambitions of the MIT/MGH CATCH program, among others (disclosure: I’m a co-founder).

One of the challenges, however, is trying to understand the quality and value of the wearable data now captured.  To this end, it might be useful to consider a evaluation framework that’s been developed for thinking about genomic testing, and which I’ve become increasingly familiar with through my new role at a genetic data management company.  (As I’ve previously written, there are many parallels between our efforts to understand the value of genomic data and our efforts to understand the value of digital health data.)

The evaluation framework, called ACCE, seems to have been first published by Brown University researchers James Haddow and Glenn Palomaki in 2004, and focuses on four key components: Analytic validity, Clinical validity, Clinical utility, and Ethical, Legal, and Social Implications (ELSI).   The framework continues to inform the way many geneticists think about testing today – for instance, it’s highlighted on the Center for Disease Control’s website (and CDC geneticist Muin Khoury was one of the editors of the book in which the ACCE was first published).

Analytic validity refers to how well does a test measure the parameter it’s supposed to measure; in the context of digital health, this might mean whether a pedometer accurately measures the number of steps, for example, or whether a heart rate monitor accurately and consistently captures the true number of heartbeats.

Clinical validity describes how well a test measures the outcome of interest – for example, if a test suggests you have a sleep problem, how likely is this to be true (essentially, the positive predictive value), and how likely is a negative test likely to be true (the negative predictive value).

Clinical utility, as Haddow and Palomaki write, “defines the risks and benefits associated with a test’s introduction into practice.”  In other words, what’s the impact of using a particular assessment – how does it benefit patients, how might it adversely impact them?  This may be easiest to think about in the context of consumer genetic tests suggesting you may be at slightly elevated risk for condition A, or slightly reduced risk for condition B: is this information (even if accurate) of any real value?

Ethical, Legal, and Social Implications remind us of the potential unintended consequences of testing or monitoring (e.g. there are obvious concerns about being perpetually monitored, for example).

This framework helps us understand why many healthcare professionals may be reluctant to welcome wearable data into the electronic medical record.  Let’s consider an example such as activity monitoring – the kind of information you might get from devices such as a Fitbit, although this example is entirely hypothetical and explicitly does not refer to any specific brand or product.

First, you might reasonably wonder how accurate an activity monitor is – how well does it measure the number of steps, and how consistently?  If you take 1000 steps, will the device always record about the same number, or might it be far off?

Second, as a physician, you might ask how well it measures the parameter you’re interested in – say activity.  Do high measurements always correspond with high levels of activity, and do low measurements always mean you’ve been idle?  (You can think of this as asking how confident should you be that a patient with low activity should be diagnosed as “indolent,” or one with high activity should be diagnosed as “active.”)

Third, what is a healthcare professional supposed to do with this information?  Operationally, what do you do with months of activity data?

Fourth, what are the unanticipated consequences of including activity monitoring in the medical record?  Can a doctor be sued because a patient’s exercise pattern changed and the physician never acted upon this?  Would patients who didn’t want their activity to be constantly monitored be subject to higher insurance rates?

One of the most interesting questions likely to emerge from the discussion of wearable data is how much does data quality matter?  One view – which I’ve heard expressed with particular eloquence by MGH clinical investigator Bill Crowley (disclosure: a former colleague and long-time friend) – is that high quality phenotypic data are absolutely essential for clinical care and especially medical discovery.  Without obsessive attention to the way data are collected, you just have garbage in, and get garbage out.  In this view, the key to success is rigorously training clinical investigators, and using carefully validated measurements.

The other extreme, which Stanford geneticist Atul Butte is perhaps best known for advocating, is what might be called the data volume perspective; collect as much data as you possible can, the reasoning goes, and even if any individual aspect of it is sketchy or unreliable, these issues can be overcome with volume.   If you examine enough parameters, interesting relationships are likely to emerge, and the goal is to not let the perfect be the enemy of the good enough.  Create a database with all the information you can find, the logic goes, and something will emerge.  Explains Butte, via email,

“For me as a data scientist, and with my style of research, I would rather take 10 data sets of ‘mediocre’ quality than 1 data set that someone says is perfect.  Those data sets of ‘mediocre’ quality aren’t usually that bad, they are just perceived as being bad….  Data quality is always improving.  Next year’s data is going to be better than last years.  But we will always find some way to criticize data.  So it never makes sense to me to wait for perfection with data.  The idea instead is to make the most of the measurements you have in front of you, right now.”

I suspect that the routine use of wearable data by the medical establishment will closely parallel that of genomic data: everyone will agree that it’s interesting, and represents an area that should be followed closely, but relatively few pioneers will actually jump in, and really start collecting data and figuring out how all this works; the return on investment will be hard to define, the uncertainty viewed as too high.

It wouldn’t surprise me if many of the same innovators that are early adopters of genomics (e.g. pursue whole genome sequencing on an ambitious scale) will be also be the earliest adopters of data from wearables, with the idea that the combination of rich genotype plus rich phenotype is likely to be an important source of insight  (again, keep in mind that I work at a genomic data company).  Within pharma, I’d suspect many of the largest companies (playing not to lose) will pursue lightly-resourced exploratory projects in this area, while companies I’ve called mid-size disruptors are more likely to take a real run at this, as part of a more confident and aggressive strategy of playing to win.

I’m obviously a passionate and long-time believer in the value of collecting and colliding large volumes of data, but I also recognize that this remains largely an unproven proposition, and I can understand why anxious administrators, prudent physicians, cautious corporations, and sensible investigators might prefer to place their bets elsewhere at the moment, deciding it’s still too early to jump in.

The skeptics may be right – but they may also arrive late to an amazing scientific party.

Spread the love

10 replies »

  1. It can be convenient when data from a medical device directly integrates with your personal electronic health record. drchrono’s EHR encouraged me to use Google Glass to transcribe my patient experience directly into the EHR! I was able to pay more attention to my patients and less on charting…

  2. The bright future will come when the patient encounters are videotaped and entered in the medical record like Youtube. I give it 18 months. That, and UCHIP – universal chipping of all patients – will give us what we are asking for.

  3. David… I share your enthusiasm about the future of Big Data analytics in healthcare. With the advent of accountable care organizations (ACOs), patient portals, social media, technology-enabled medical homes, telehealth, remote patient monitoring, wearable devices and genomics, the volume of healthcare data at the disposal of healthcare providers and patients will grow exponentially in the years ahead. This will provide a rich source of valuable information to better manage health on behalf of the patients we serve, including in areas that are major determinants of health that have largely been out of the reach of clinicians, e.g., behaviors, genetics and environmental factors. Certainly, there are challenges to overcome and issues to manage, but the potential for an amazingly bright future for healthcare is very real and very exciting.

  4. So many questions raised, so little capacity on the internet…

    This article raises many interesting pros and cons that may never be satisfactorily resolved. It is also the first time I’ve seen ACCE applied in any kind of population health or even patient health discussion. Since I had never heard of it, I doubt any population health vendors have either. And yet each of those 4 initials themselves could support a PhD thesis (one that would be obsolete upon completion).

    As David concludes rather indisputably, there is no clear answer — but that doesn’t mean we should shy away from pursuing one. What will definitely be the case is that on the path to the right answer there will be plenty of obviously wrong conclusions reached by people out trying to make a buck on this, allowing me to make my buck by pointing out what idiots they are.

  5. The answer to this question depends in part on, Whose EHR? Although there probably should be, there is not yet a single EHR for each person. When there is, we could possibly discuss where personal information like this might reside, how it might be summarized for quick overview viewing by clinicians, and so forth.

    But in our current state of numerous EHRs, belonging to each and every clinician, hospital, etc., this question is too vague. Should personal fitness data go into the primary care EHR? Since that may be a physician with limited resources, should we expect his or her system to be able to accommodate such data? In our still predominantly sickness-based health care system, how much time and effort should we expect that physician to devote to it?

    This is a complex question whose answer depends on the underlying context.

  6. Does wearable data belong in an EHR? Yes, eventually. Medicine is undergoing a major shift because of information technology. And by information technology, I am not referring to just EHR systems, but everything that enables the collection, searching, and analysis of data. Clinical care has always been an information intensive field, but for most of its modern history paper and brains have been the only to information management tools available.

    Fast processors, sophisticated databases, high-capacity storage, and fast networks are relatively recent occurrences in the history of health care. We are still ironing out information and data exchange standards. Information technology has advanced faster than our ability to incorporate its capabilities into routine care.

    Dumping data from wearable devices onto already stressed providers is a bad idea because current EHR systems are not ready to manage what they would receive. Clinical decision support is still primitive. Data analytics is more often said than done.

    Fortunately, there is a general acceptance that information technology is a part of clinical practice. With this acceptance, we can now turn our attention to making software that better supports clinical care. And by better I do not mean adding a few new features to systems designed to be patient data repositories (EHR systems) and declaring them to be the solution, but rather designing systems from the ground-up to intimately support clinical work (clinical care systems).

Leave a Reply

Your email address will not be published. Required fields are marked *