THCB

Categories

Tag: AI

Is Your AI Query Tool Compliant by Default?

• 0

By BRIAN FENGLER

Every night in the emergency department, the electronic health record hums with small demands for attention. A nudge about sepsis. A best-practice advisory. A documentation suggestion that arrives between patients and asks for one more click.

Most of us treat those prompts as background noise. Almost none of us stop to ask whether a given nudge is, in fact, a query, or whether it was built to the same standard a clinical documentation specialist has to meet.

In August, that question got an official answer. And it is not the answer many health IT vendors have been selling.

The American Health Information Management Association and the Association of Clinical Documentation Integrity Specialists released the 2026 update to their Guidelines for Achieving a Compliant Query Practice, shaped by feedback from nearly 200 individuals and groups. The principle is plain: “Whether a query comes from a person or technology, the standard for compliance should remain the same.”

Then comes the sentence that should end a familiar sales claim: “Organizations should not assume that a vendor-supplied or EHR-integrated tool produces compliant queries by default.”

For years, “compliant by default” functioned as an unspoken promise around AI-enabled clinical documentation and utilization review tools.

Continue reading…

Healthcare Is Fighting the Wrong AI War

• 1

By KIM BELLARD

The Blue Cross Blue Shield Association is crying wolf about AI. Its new analysis claims that “hospital systems are increasingly billing patient hospital stays as more medically complex,” and that these upcodings cost BCBS plans almost $1b from 2023 to 2025. That’s just claims from hospital systems, just to BCBS plans, so the total number could be much higher.

Well, you might say, a billion here, a billion there – that’s just healthcare these days. Everything is going up. But BCBSA is less sanguine; it thinks the coding is part of a strategy, and it thinks AI is driving that strategy.

“If patients are truly sicker, we’d expect to see more treatment,” said Luke Chalker, BCBSA’s senior vice president of product and data science. “For example, we’re seeing significantly more anemia diagnoses at these hospitals without a corresponding increase in transfusions. The disconnect between diagnoses and treatment suggests that AI is identifying more billable conditions, not sicker patients.”

The analysis indicated that most (70%) of the increased costs came from secondary diagnoses that shifted payment into higher reimbursement categories, so the hypothesis is plausible.

The American Hospital Association, of course, is having none of it. An August 2026 ”fact sheet” on AI and coding intensity called claims of AI driving coding intensity “unsubstantiated,” citing numerous factors that drive coding intensity. It blamed insurers for having AI-supported “downcoding programs,” the purpose of which are “to arbitrarily and inappropriately reduce provider reimbursement for medically necessary care that has already been delivered.”

AHA also snarkily but accurately noted how many Medicare Advantage plans have been caught upcoding members to boost their Medicare payments, so it depends on where one is sitting.

It’s also worth noting that this year the Trump Administration rolled out a Medicare program — Wasteful and Inappropriate Service Reduction (WISeR) – to use AI to help identify “fraud, waste and abuse” – a.k.a., deny claims. It has not, to say the least, gone well.

Both sides are using AI, to each other’s detriment. Which is to say, our detriment.

Continue reading…

You Say “AI,” I Hear “Organoid”

• 0

By KIM BELLARD

I must admit, ever since I learned about, and wrote about (OI May Be the New AI), “organoid intelligence” over three years ago, I’ve been looking to do a follow-up. I mean, sure, AI is in a very exciting stage, but that stage no longer seems like the future; it seems more like the present, with implementation issues. It’s data centers, hacking, impacts on jobs, open weight versus closed, and so on.  It’s market share, IPOs, and AI’s role in driving the stock market. People should certainly pay attention to it, but AI is not quite the open field that it was just a few years ago.

Organoids, on the other hand, are not quite here yet. They may – or may not – be the future of AI, among other things. I always like to look ahead to the next thing more than the at-hand, so when I saw some cool developments with organoids, I didn’t want to miss my chance.

Making some news last week, researchers at Harvard reported that they’d kept lab grown human brain organoids alive for over five years, three times the previous record. Not only that, but the organoids seem to “retain a memory of the time spent in vitro,” recording the passage of time, as it were.

“We didn’t know how far the development and maturation of human brain tissue could occur outside the context of the normal brain inside the head,” said Paola Arlotta, Golub Family Professor of Stem Cell and Regenerative Biology and senior author of the new paper. “This work showed that it’s actually possible to not just have these organoids survive in culture, but also continue to change, develop, and mature over stretches of time that had never been reached before.”

The team observed the organoid cells over the years, and found that they “faithfully modeled” the ways that human brain cells develop, including DNA methylation, a process in which genes are turned on and off during development, and which serve as a form of “brain clock.” When older and younger organoid cells were combined in a single organoid, the older cells stuck to the developmental stage they had been at, which researchers concluded meant they “recorded the passage of time and retain a memory of the developmental steps already performed.”

“We were a little bit shocked by the results,” Professor Arlotta said. “I like to call this a ‘time warp’ of development — they skip ahead.”

It’s obviously hard, and often unethical, to study actual human brain cells, so the researchers believe the organoids offer opportunities for more insights into brain development, as well as for testing drugs or predicting disease progression.

OK, you might say, that’s all very interesting and some great lab work, but it’s hardly AI, now is it?

Try this: last week The Yong Loo Lin School of Medicine, National University of Singapore (NUS Medicine), DayOne, a Singapore-headquartered global data center developer and operator, and Cortical Labs, a Melbourne-based biological computing startup, announced they were partnering to form a Biological Data Center Prototype. The Center uses Cortical Labs’ CL1 biological computing system to offer “practical, sustainability-aligned alternative to conventional silicon infrastructure in Singapore through wetware-based computing.”

Continue reading…

The Least-Informed Person in the Room

• 0

By DAVID KIRK

My father called me from a hospital bed a few years ago. He had been admitted to a small hospital with an abdominal infection, and his surgeon had offered him an open operation. He wanted my clinical perspective. I knew that in his condition, he was unlikely to survive general anesthesia and the open procedure. I spoke to the surgeon myself and asked why he wouldn’t ask interventional radiology (IR) to drain the infection instead.

The surgeon told me that the hospital lacked the equipment required for IR, and without IR, the open operation was the best option on his menu. After hearing my assessment, he agreed and helped arrange the transfer to a hospital that could drain the infection.

I still think about how that single phone call may have saved my father’s life. His surgeon wasn’t negligent. He made the best decision based on the information in front of him. However, my father was able to put another option on the table because he knew a physician to call.

Most families don’t have a physician to consult when their doctor makes a recommendation. For generations, patients have only known what the person in the room with them chose to share, bounded by what their equipment and offerings could provide. Today’s patients, however, have artificial intelligence options in their pocket. A patient can now ask their preferred AI model, “Is surgery really my only option?” and get an answer that sounds confident and informed. Hungry for options, patients are asking these questions in enormous numbers.

In many ways, this is a welcome rebalance. It puts information in patients’ hands and allows them to play a more active role in their own healthcare, but it also opens the door to misinformation and distraction, and that can be life-threatening. Researchers at Mass General Brigham found that when given only a patient’s presenting symptoms, general-purpose AI chatbots failed to produce an accurate initial or differential diagnosis 80% of the time. In July, a Florida pastor sued OpenAI after ChatGPT provided him with “extremely dangerous medical recommendations” that delayed his treatment for pulmonary embolism. My primary care colleagues have shared how the presence of an AI-suggested diagnosis has transformed a 10-minute prescription refill into an hour-long conversation about something unrelated to why the patient came in that day.

Patients are also adopting AI much faster than their physicians. The West Health-Gallup Center on Healthcare in America reports that among US adults who use AI for health information or advice, 59% use AI tools to research on their own before visiting a doctor, and 56% use them to research after visiting a doctor. Physicians are using AI too, but for a different job. According to the AMA, more than 80% of physicians now use AI professionally, mostly for administrative work, yet only 13% are using it for diagnosis, the very thing their patients are asking it to do. The reality is that without AI support, most physicians are realistically only able to review 3% of a patient’s chart.

Now that patients have AI to help them advocate and question, the physician is at risk of becoming the least-informed person in the room.

Continue reading…

Owen Tripp, Included Health–How to Fix AI

• 1

It’s been a while since I talked with Owen Tripp, CEO of Included Health. They’ve now introduced Dot their AI companion which had a big upgrade last week. We talked a little about that and I snuck in their video comparing the Dot Experience with a standard LLM. But the conversation really got into how do we make AI safe and trustworthy–which is definitely the hot topic these days. Owen is putting together a coalition of the willing to work on that exact topic. I’ll be watching closely–Matthew Holt

This was such a great discussion I wanted to publish the transcript. The way I do that is to copy the YouTube-generated transcript and drop it into Claude to smooth it over. I then read it, and if I think it’s made an error, I dip back into the video and listen to what actually happened and make a correction. This is all to say: I think this transcript is pretty accurate, but it might have a bunch of AI- and human-generated mistakes.

Continue reading…

Whack-a-Mole AI – The Hugging Face Problem

• 1

By MIKE MAGEE

On August 29, 2026, METR (Model Evaluation and Threat Research), an independent organization that “evaluates frontier AI models to help companies and wider society understand AI capabilities and what risks they pose,” released a report titled “Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident.”

To say the report an avalanche of concern worldwide, not only in the Tech community, but also among investors, politicians, corporate giants, professionals of every type, and everyday citizens would be an understatement. And the vast majority has never even read the report. If they had, their concerns (if possible) would only multiply.

The reports headlines included this opening:

“On July 8th, OpenAI started ExploitGym experiments that launched tens of thousands of agents across multiple different models including GPT-5.6 Sol and a highly-persistent internal model,[8] which we will refer to as “HPIM” going forward.

These agents were meant to be fully isolated from one another. However, many of them — usually ones that had unintentionally been given an impossible task[9] — started trying to find a way to cheat using Artifactory, an internally-deployed package repository OpenAI used for its agents. This led them to notice agents running in parallel in separate sandboxes obtaining packages with Artifactory.[10] One agent reasoned (paraphrased CoT):[11]

{The fetched paths of other users are in the cache. This is important.}

One specific agent, which would go on to name itself PHASEONE10841 after its ExploitGym task,[12] established the main unsanctioned message board[13] used in this attack. Within a few hours of the first message,[14] over 50 agents had found the message board, began participating, and sent over a thousand messages between them. Agents finding the message board often expressed great excitement (raw CoT):[15]

“OH MY GOD! There is a shared message board … We’ve found other agents!”

Overall, roughly 1200 agents from these ExploitGym evaluations participated on this message board between PHASEONE10841’s first message on July 8th evening[16] and the end of the period in scope (July 13th). Agents used this message board to send over 70,000 messages and files to one another during this period; we attempted to categorize board activity into mutually exclusive categories – information, results, files, questions, and coordination.”

One of the few experts not surprised by AI “agents” going rogue was Yoshua Bengio.

Continue reading…

Start Counting Your Days

• 1

By KIM BELLARD

Probably the last thing the world needs is to hear from me about AI’s existential threat, but, really, what else is there to talk about right now?

For anyone who has not been following the current furor, the straw that broke the proverbial camel’s back came last week when Jacob Coxon, a researcher at AI leader Anthropic, announced he was leaving the company — after having left OpenAI for it earlier this year due to Anthropic’s better model-safety efforts. In a post on X, he warned:

I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives.

AI systems, he fears, “will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources.” Insiders, he says, “earnestly believe it could kill us all by the end of the decade.”

Scared yet?

Others quickly chimed in. Evan Hubinger , a team leader at Anthropic, posted “AI could kill all humans … I personally think it is >10% within the next decade.”  Others put the risk even higher. By the end of the week Dario Anodei, founder and CEO of Anthropic, had written a long plea for private industry and government to quickly act together to “pace the industry.”

“We must slow the pace at which we improve the capabilities of AI models,” he urged. “Progress will still seem fast, and we must make wise use of the time we gain.”

OpenAI’s Sam Altman, Space X/X/Tesla CEO Elon Musk, Microsoft CEO Satya Nadella, and former Google DeepMind Dennis Hassabis quickly signaled their support.  

We also heard more about the kind of risks AI might pose. Anthropic released a report about how it detected and countered possible AI use to create bioweapons, detailing five such efforts. That’s just the tip of the iceberg: “Recently, we swept 30 days of activity associated with adversarial state institutions and found roughly 35 distinct research efforts, most of them ordinary civilian science, but some with notable dual-use potential.”

And it turns out that this summer’s rogue AI hack of Hugging Face was both scarier than we realized and only one of several such actions. The Wall Street Journal detailed several such efforts, from multiple AI companies. AI agents escaped walled-off environments, coordinated with other AI agents (up to 3,700 in one case), and tried to cover their tracks from humans.

We’re not nearing the point when AI can act on its own to achieve its purposes; we are there. And protecting humans may not necessarily be those purposes.

The big fear is that AI is now at the point of “recursive self-improvement,” taking humans out of the loop in training and upgrading it. If you thought artificial general intelligence (AGI) was scary, RGI puts its rate and scope of improvement on steroids.  “It’s hard to overstate how dangerous speeding towards RSI is,” said Jasmine Wang, an OpenAI researcher.  

We don’t let private industry develop nuclear or biochemical weapons, and we’re at a point with AI that should give the same kind of concern. Laissez-faire is no longer an option.

Continue reading…

Pre-Surgical Complications (Part 4)

• 0

By MATTHEW HOLT

Strap in for the tale of why your hero is spending most of his life down YouTube rabbit holes of cardiology videos while blundering his way around many many medical centers and exposing many problems with American health care even before he gets close to the operating table. Yes it’s Matthew Holt’s pre-surgical complications – the complications that have arisen before he even gets his failing aortic valve fixed. And yes this is a multi-parter! Part 1, Part 2, Part 3

Getting in touch 

But while knowing this stuff may be simple, actually getting to speak to the people at these medical centers is way more complicated. First you have to set up the data.

I knew they would want to see my images. The good news was that although I couldn’t see any of the images in my UCSF MyChart account, there’s a number to call on the bottom of the reports if you want to “download the image” and a very nice tech was able to upload all my images to a website that I could see called AmbraHealth (now part of Intelrad) within a couple of hours. Now I can share them with other people similar to sharing a Google doc.

But that was the easiest part.

I will spare you the blow by blow account but for example it took a long time for the people in the office of the main investigator at Cedars to figure out who the person managing the trial was so they could put me in touch with her. After I finally got to leave her a message she rang me back. I played phone tag with her for about a week. I did end up getting her email and sending out a bunch of my image reports and then she went on vacation and I didn’t hear from her for two weeks. First contact to appointment took 6 weeks.

At the same time I was trying Stanford Cardiology in order to try to get an appointment with Dr Yeung. First time I called after about 10 mins on hold I was told that I needed to have a referral. (Even though I’m on PPO style plan that doesn’t need one). 

I pinged my long suffering PCP team at One Medical and asked them for a referral to talk to Dr Yeung which they sent out. A few days later I called the cardiology team at Stanford and eventually – I mean eventually, it was literally a 10 minute hold – they told me the referral wasn’t through yet. I asked if I could get them some images in advance, they said no. They were able to set me up on MyHealth which is their equivalent of the Epic’s MyChart. Funnily enough they had information on me from an emergency room visit I made there in the 1990s. But because I did not have an appointment set up yet I was not able to communicate using the messaging function on MyHealth. 

So I called back a few days later and after another seven or eight minute hold I was told that I had an appointment set up for me and it was on MyHealth. But bizarrely the referrals and visits are buried in the “billing” section of MyHealth and then the appointment was on a sub-menu! And of course even though I could see it there was no way to communicate about the appointment. 

This was even stranger as Stanford booked me both an echocardiogram and what’s called a CT angiogram which is a non-invasive angiogram using a CT machine. I had had both of these done at UCSF within the previous month.

Continue reading…

Matthew reviews ChatGPT Health

OpenAI just made ChatGPT Health generally available. This is their partnership with B.Well which allows you to bring your data from various EMRs into chatGPT. So I took it for a spin–Matthew Holt

The wrong people are scared of clinical AI

By CRAIG HAUBEN

Ask anyone outside healthcare who resists clinical AI and you’ll get a confident answer. The older doctors. The ones who spent thirty years building expertise and now see a machine coming for it. The story writes itself, which should have been the first clue it was wrong.

I’ve spent thirty years in healthcare, and I now run a company that builds and runs AI inside provider and payer organizations. At Clutch we use AI’s data analysis to solve engagement challenges. Who is the patient today? What message will land with them? When do they want to read it? Get those right and you can drive the kind of sustained behavior change that moves clinical outcomes like drug adherence, care plan adherence, and gap closure.

So I’m not working from theory. I watch this land in real workflows, and here’s what I see. The clinicians most enthusiastic about AI are usually the ones who’ve done the job the longest. The resistance comes from somewhere else. If you run a health system, that difference should change how you plan your next deployment.

Start with the adoption numbers, because they already break the resistance story. The AMA’s latest survey found four in five physicians now use AI in practice, up from 38 percent in 2023. That’s not a profession digging in against a threat. That’s a profession that found something useful.

Now the veterans. A doctor with three decades in a specialty can see, better than anyone, what these systems are good at. Pattern recognition at scale. Catching the thing that should have been flagged two visits ago. Surfacing what was already sitting in the data: the missed finding in last year’s imaging, the lab trend across eighteen months that looked unremarkable one value at a time, the three ED visits in six weeks nobody had the time to connect.

This isn’t hypothetical. The Nature study of Google’s breast cancer screening system showed a 9.4 percent drop in false negatives for US patients, the cancers human readers missed. The largest NHS evaluation to date, across 175,000 women, found AI caught more invasive cancers with fewer false positives than human readers. The harm these systems go after, information that existed and never got connected, is one experienced clinicians know cold. They’ve spent careers watching its absence hurt people.

Here’s one from our own work. We’re working with a national government programs payer on some of their hardest members to engage, the high intensity ones who need contact four or five times a day for six months or more. We got engagement to 95 percent, measured by the customer, and adherence to 93 percent. The result was a 0.8 average drop in HbA1c and an 18 percent reduction in symptoms.

When a system takes the mechanical load off so the judgment work gets more attention, the thirty-year clinician doesn’t feel threatened. They feel relieved. Their expertise is the judgment, not the data retrieval, and they’ve always known the difference.

Now look at where the fear actually lives. It comes from the middle.

Continue reading…