Medicine Is a Profession That is Rapidly Losing Control of Its Tools.

Artificial Intelligence hype and reality are everywhere. However, the last month or two has seen some thoughtful reflection. HHS / ONC announced “Hype to Reality: How Artificial Intelligence (AI) Can Transform Health and Healthcare” referencing a major JASON report “Artificial Intelligence for Health and Health Care [PDF -817 KB],”. From a legal and ethical perspective, we have a new multinational program: “PMAIL will provide a comparative analysis of the law and ethics of black-box personalized medicine,…”. Another Harvard affiliate writes “Optimization over Explanation” subtitled “Maximizing the benefits of machine learning without sacrificing its intelligence”. Meanwhile, an investigative journalism report from the UK “Google DeepMind and healthcare in an age of algorithms”, “…draws a number of lessons on the transfer of population-derived datasets to large private prospectors, identifying critical questions for policy-makers, industry and individuals as healthcare moves into an algorithmic age.”

What sets these major papers apart from the constant AI and Big Data science or hype is that they are not particularly technical. Although long, they are accessible to pretty much anyone including physicians and patients. What shocks me, as a career medical technology engineer also trained as a physician, is how passive physicians and patients seem as medicine shifts from open public science to secret corporate black boxes.

Medicine is a profession that is rapidly losing control of its tools to corporate and political interests. First, they came for the information itself and introduced electronic health records that doctors do not purchase or control. Some doctors, frustrated by the loss of control, retired but most just complained and learned to type faster. Now they’re coming for the very core of the profession as a “learned intermediary” trusted by the patient. Black box AI may pay lip service to the physician as an intermediary but everyone understands that it’s the secret and proprietary machine that’s doing the heavy lifting. In machine we trust.

From a patient and physician perspective, two things define our loss of control and shifting trust: health records and secret methods. Patients have lost control over health records. Since 2002, corporations are able to use our health records without our consent. The trend to involuntary and hidden surveillance of everyone continues and grows to nationwide scale with the new HHS administration. This, we’re told, is the price we must pay to keep us safe and make research, including machine intelligence, effective.

Secret methods, we’re told, is the price we must pay to bypass the limitations of human understanding. Asking the AI to explain itself or making the AI an open public good like medicine used to be would simply slows progress and costs lives. Patients lose control of the records that train the AI. Physicians lose control of the open science that used to be the foundation of trust in the profession.

I firmly believe in the power and promise of technology, including AI, but we must not cede control out of greed and frustration.  As the JASON report clearly states, EHRs don’t have access to the full range of trusted information the AI needs. We can build a national health surveillance bureaucracy like European countries have but it’s unlikely to have access to our cell phones and the social determinants of health. We can also regulate black box AI at a corporate level but that takes the power of innovation and experimentation away from the licensed clinician.

AI is becoming an essential part of medicine. Patients need to trust their most sensitive data to the machines the way we used to trust our physicians before institutional EHRs and national-scale data breaches. Doctors need to remain the learned intermediary by keeping full transparency and interactivity with the machine. Secret software and black-boxes do not respect the role of physician as the root of trust and decentralized regulation. The current approach to AI is not in the public interest and the medical profession needs to snap out of its daze.

Adrian Gropper, MD, is the CTO of Patient Privacy Rights, a national organization representing 10.3 million patients and among the foremost open data advocates in the country.

Categories: Uncategorized

5 replies »

  1. If AI becomes its most glorious, it will be a “thinking” being. I.e. we shall be able to allow it to read Harrison’s Textbook of Internal Medicine and it will be able to suggest the symmetrical arthritis we are seeing in this particular patient is because of Mediterranean Fever. Wow. It may not be able to specify and print out for us its thinking, or algorithms, anymore than a creative person could tell us why she is making a chess move…or has a hunch there is a dangerous man in the elevator.

    But, as AI becomes a real being and a tool too, it is a de facto “consultant”. Therefore It has to be a mamber of the medical staff. [Sic!] AI shan’t be allowed to process personally identifiable information, PII, unless it is a member of the medical staff and it cannot give outcome information unless it is a member of the medical staff. It cannot practice medicine without a license. As Adrian and Deborah both aver, this is dangerous stuff. The workings, both input and output have to remain top secret as long as it is working on PII….else AI becomes a tyranny of management. Just think of all the ways it could control health care!

    It has to remain a tool of doctors or we will rue this day.

    And Adrian is again on target when he worries we are losing it.

  2. The key thing I would add to Adrian’s brilliant essay is: AI does NOT have to be secret. In fact, secret AI should be illegal. And your pii should never be collected w/o meaningful informed consent, transparency and accountability, these are absolute requirements for trust in AI. If AI for ‘healthcare’ so great, everyone would agree to help. (key disclosure: Adrian Gropper MD is the CTO of Patient Privacy Rights)

    Autonomy, choice, and self-determination are the fundamental human rights required for freedom and Democracy, not just for the practice of Medicine.

    The context we live in today matters, every generation has to work to preserve our rights to Democracy—individuals have strong rights to a private life (see Brandeis’ famous law review article, it’s only 12 pages: https://groups.csail.mit.edu/mac/classes/6.805/articles/privacy/Privacy_brand_warr2.html).

    We’re constantly surveilled and the traces of our pii are collected everywhere in our homes, and everywhere we go. Just because we can’t see or feel our data being taken away, we sense it, and can’t prevent it.

    Massive deceptions are built into how AI is portrayed, actually “marketed” is the correct word, by the US “healthcare” industry:
    • as “magic” that can’t be understood or examined, but never-the-less is accurate, infallible, and far smarter than people
    • and the extremely misleading idea that there is no way to do AI ethically, effectively, or openly.

    AI in healthcare is not magic—and much of what the healthcare industry calls “Big Data” and “AI” are NOT ‘intelligence’ either, but simply trendier names for ‘pattern matching’, which humans are actually pretty great at too. During the millennia the profession of Medicine developed, how do you think doctors identified new diseases and effective treatments? They were great listeners, observers, and examiners who generated great theories: they were pattern-matchers. Still are. They did it by paying close attention to patients, to people willing to trust them because of their ethics: the physician’s job is to put the patient’s needs first, to do no harm, and keep their sensitive medical records private (not disclose them w/o meaningful consent).

    Do you trust people who betray you? Who share intimate information about you? The profession of Medicine has come so far ONLY because so many sick people trusted physicians who were ethical, caring, and took responsibility for their health and lives.

    Current so-called ‘AI’ in any realm cannot actually replicate real intelligence or genius…yet. “Intelligence” means the kind of genius ideas Einstein could dream up, or Watson and Crick, or lots of others: Edison, or Da Vinci, and many others. ‘Intelligence’ means creative, inspired often instantaneous new connections and paradigms far, far beyond and different than pattern matching’s capabilities. [SEE the work of Vikash Mansinghka at MIT—he gave a brilliant keynote at last year’s summit he explodes the fake news about what AI is and is not: https://pprmedia.streaming.mediaservices.windows.net/91b0d6c7-4d11-4332-a725-9175b89aa500/11-2017HPS-Keynote-AugmentingHum_1280x720_4500.mp4%5D

    Any ‘AI’—if it is in fact AI, as opposed to pattern recognition — when used for any endeavor that affects human life, YOUR life, any human being’s life cannot be secret. What if the AI software that flies airplanes was secret and proprietary? Guess what? It’s NOT, for obvious reasons—Every airplane has a black box that records all the transactions, events, and steps the software takes, in order to make sure that ERRORS/failures/and deaths can be prevented. AI for health has to be able to record every step, every decision, every bit of what leads to an outcome. We must be able to name, know and be able to uncover the hidden biases built in by the human coders. The coders all start with assumptions—aka biases, and these must be transparent.

    • Another example of AI transparency—the 3 US credit rating agencies must expose their AI algorithms every year so outsiders can examine what processes were used to predict credit scores. Biases are discovered and the system becomes fairer every year.

    Anything that that can kill people MUST be open and not secret. ALSO, the foundation and ethics of trust in Medicine and physicians are altruism and sharing knowledge for the greater good. We must treat poor people as well as we treat the rich. All beneficial discoveries and cures MUST be available to everyone.

    If so-called “learning-AI” systems cannot track or reproduce the steps that lead to a treatment decision or outcome, then we should step back and start over. “AI” for our bodies and minds cannot be secret—it must be open for everyone’s benefit, and our data must be collected openly, with meaningful informed consent. Why should data about our minds and bodies enrich IBM, or Google, Amazon, Facebook or any giant corporation?

    I’ve practiced 40+ years, and recall when Medicine, hospitals, and pharmaceutical companies were truly not-for-profit (‘70s)—the aim was to make 4% revenue above costs. Today huge corporations and monopolies focus on maximizing profits, not healing the sick. They extract billions from patients (all of us) and the results are terrible. We have worse health and shorter life spans compared to rest of the First World, Western democracies.

    Do you believe that younger physicians, raised on EHRs and hired by corporations, feel the same deep, personal obligation to patients that used to be the physicians’ hallmark? They no longer think of patients as their responsibility, they feel no ethical obligations—most don’t keep personal copies of their patients’ complete health records, because they believe it’s the hospital’s job or the EHR vendor’s job—even though state laws all require physicians to keep complete patient records for a period of years.

    • Not keeping your entire records of treatment is a key sign your doctor is not protecting you and doesn’t feel responsible for you. Younger physicians think the corporations they work for are responsible for record-keeping.

    Terrible HIT systems and corporate healthcare culture are key causes of physician demoralization and abandonment of direct patient care. Physicians report spending 52% of their time on computers. The most highly trained people in ‘healthcare’ spend half their day as data entry clerks, not practicing Medicine.

    • Do you really think IBM or Google care about you and will work to save and comfort you if you are very sick?
    • Do you believe your health plan, a hospital chain, or the drug industry will go to the wall for you?

    The US ‘healthcare industry’ literally disconnects patients from doctors. This sometimes deeply intense human dyad is critically needed when you are very scared and very sick. AI & robots won’t cut it.

    • As of 2016, fully 89% of US patients withheld information from providers because they don’t trust technology or how the doctors use it: https://blackbookmarketresearch.newswire.com/news/healthcares-digital-divide-widens-black-book-consumer-survey-18432252

    So much technology has seriously harmed the US and our healthcare system. But re-designing AI to support the patient-physician relationship, ensure human rights to privacy, and improve treatment could begin to restore public trust in doctors and technology. Addressing and fixing one of the scariest new technologies would be a great move.

  3. Some of us have been complaining about the inherent surveillance mechanisms built into structured-data EHRs for a very, very long time, and sad to say, some of us have vague memories of being ridiculed and impatiently dismissed by people who now seem so woke to the dangers of what that surveillance net has begotten.

    The entire narrative about giving data to patients who control it and to doctors when they need it and such other nonsense is and always was a ruse. Data is exclusively owned and controlled by those who own the database where it resides, and those owners are in this business to make boatloads of money.

    We can moan and groan from here to eternity, but unless we manage to pass laws that protect our privacy for real, and prohibit exploitation of our digital selves (even with so called consent), under heavy financial penalties and decades of hard-labor imprisonment for corporate executives, nothing will change. Which means that nothing will change.

  4. Let’s see: the Academy of Health is under the mantra of Congress, the National physician specialties evolve under the mantra of the Association of Medical Specialties; the legally separate Medical Staff of each hospital is under the mantra of their hospital’s parent enterprise; the medical schools survive under the mantra of the Association of American Medical Colleges; Primary Healthcare is under the mantra of it underfunded codependency with the sources of its financial support; Complex Healthcare is under the mantra of its institutional market-share; AND there is no connection between the availability and accessibility of Primary Healthcare and its needs within each community.

    Overly simplified you say, maybe so, maybe not. Meanwhile, our nation’s excessive health spending produces @80% of our nation’s annual Federal Deficit, and our nation’s decline in each community’s COMMON GOOD and its associated level of Social Capital contributes to an unconscionable worsening level of maternal mortality in association for 50% or our citizens. The level of Paradigm Paralysis is so entrenched that the issues of AI cannot be accommodated while under the guise of a unified electronic medical record.

    Back at the ranch for our own nation, there is no current commitment to apply the world wide, developed nation standard of uniformly available and accessible Primary Healthcare for each citizen, community by community. One more time, its all Paradigm Paralysis.