Will Apple Track Your Mind, Not Just Your Heart?


If your heart throbs with desire for the new Apple Watch, the Series 4 itself can track that pitter-pat through its much-publicized ability to provide continuous heart rate readings.

On the other hand, if you’re depressed that you didn’t buy Apple stock years ago, your iPhone’s Face ID might be able to discover your dismay and connect you to a therapist.

In its recent rollout of the Apple Watch, company chief operating officer Jeff Williams enthused that the device could become “an intelligent guardian for your health.” Apple watching over your health, however, might involve much more than a watch.

The iPhone models introduced at the same time as the Series 4 all deploy facial analysis software. The feature works in part by projecting a grid of more than 30,000 infrared dots on the user’s face in order to create a three-dimensional map for user recognition. This Face ID feature has been upgraded from the iPhone X by addition of a new chip, the A12 Bionic, which improves the ability of the phone’s artificial intelligence “neural engine.” Reportedly, those superior smarts will show up as clearer photos and FaceTime calls due to better detection of faces and bodies when taking pictures and video.

But that improved clarity has other applications. Research shows that your facial expressions can provide the information about your emotional or mental state that’s then analyzed by “emotion algorithms” – or, perhaps, overanalyzed in the absence of other context about your life.

Meanwhile, the smartphone’s microphone and other sensors can gather additional clues. Signals collected on facial expressions, voice quality, language use, physical activity, music choice, and geographical location can all be used “to predict and prevent mental health crises.”

To be clear, research on using the iPhone for mental health monitoring predates Face ID. Although that research is continuing, there’s no public evidence as yet that facial analysis capabilities in mental health are nearing implementation. Nonetheless, as artificial intelligence capabilities continue to increase in sophistication, along with evidence linking mental health problems too expensive physical health ones, it would be naïve to expect makers of smartphones and wearables to overlook this market’s potential.

The democratization of medical information being enabled by mHealth apps could truly empower individuals, rather than merely “engaging” them to do a better job of following doctors’ orders. (Which, I hasten to add, may often be appropriate, but also may not be.) Still, going from a company that helps us play music for every mood to one that actually measures that mood presents a perilous transition.

Yes, as the New York Times wrote, the Apple Watch’s “evolution from a fitness tracker into a health-monitoring device makes it vastly more interesting in the long term.” The introduction of the Series 4 watch promptly caused Fitbit shares to plunge.  According to analysts quoted by CNBC, the new watch could take a big chunk out of sales of traditional Swiss watches.

But as more Apple devices are used individually and collaboratively with each other to continually monitor of our bodies and our minds, the focus on diagnostic accuracy of these direct-to-consumer apps will increase – a topic I’ve recently researched and written about.

Moreover, there are serious concerns about how widely Apple may be sharing facial information, whether with advertisers or others. Information is power: to what extent will Apple’s devices empower individuals, and to what extent will they empower health care providers and health insurers, an array of marketers and advertisers or maybe even government?

One example: John Hancock recently said that all its life insurance policies will include incentives for policyholders to improve their fitness through using wearable devices such as the Fitbit or Apple Watch as part of the Vitality fitness program. While that shift certainly boosts individual empowerment for health, it also gives new power via the policyholder data collected to John Hancock, a unit of Manulife Financial Group.

Another example: AliveCor just announced its own electrocardiogram reader that works with the Apple Watch and will provide information on 100 diseases, not just the one (atrial fibrilliaton) Apple has trumpeted.  Again, that information empowers consumers, but how that information is shared and its impact remains to be seen.

Famed Apple design chief Jony Ive was spot-on when he summed up the impact of the Series 4 watch to The Washington Postby proclaiming: “Every bone in my body tells me this is very significant.” However, Ive, design genius though he may be, should remember that consumer expectations about “fitness” and “health care” are very different.

Winning our hearts and minds is not the same thing as constantly monitoring them.

Michael is a consultant specializing in quality of care, patient empowerment, web-based health, and is an adjunct Associate Professor of Medicine at Northwestern University’s Feinberg School of Medicine.

Spread the love

5 replies »

  1. I like the facial mentail health aspect which will detect when a gun purchase is considered to satisfy the need for revenge.

    Will these smart devices auto-dial 911?

    • I could reply humorously. But you actually raise a serious issue: how do we act on the deluge of information that will sometimes be highly reliable and life-saving and sometimes not — and we’re not sure which. Tx.

  2. Good post, Michael. I’m now 5 weeks out of open heart aortic valve replacement surgery, so ongoing cardiac monitoring is of interest to me. I looked at a bunch of this stuff and decided “nah.”

    • Good for you, Bobby. You know, they have a way to do this–in selected cases–through the femoral artery, percutaneously. I have seen lectures on this method and it worries me because it has to leave in situ and crush the diseased natural valve and any plaque in the vicinity…and naturally some of this plaque material can fly up the carotid arteries and cause strokes. So, I think you were smart to do it by an open procedure but this larger technique does prolongs the recovery a bit. Good wishes to you for an uneventful and complete recovery. I have a little aortic regurgitation myself…nothing they can do for it. You must have had stenosis…and once you get CHF symptoms from this, they must do surgery QUICK, well, within a few months. So we are happy you did the right thing and are still with us.

      • Oh, and also the crushed natural valve can cover the ostia of the coronary arteries. Not good.

Leave a Reply

Your email address will not be published. Required fields are marked *