Book a meeting

Health-Tech AI: The Trust You Can't Borrow

Health AI keeps borrowing trust from doctors. For the people who trust doctors least, there's nothing to borrow. And that's where you find out what a product is actually worth.

by Kelly Smith • 11 August 2026

Ask why anyone should trust a medical AI, and most answers quietly point away from the machine. They point at the person behind the machine. Trust the clinician, and the AI inherits it. Build the AI into a relationship that already works, and the relationship does the heavy lifting.

It's an elegant process. It's also a loan.

A study out this month in JAMA Network Open lays the mechanism bare: thirty-four people, mapping their comfort with AI across care scenarios. Trust in the tool ran straight through trust in the clinician. The relationship was the conduit, and everyone in that study already had a clinician they trusted. The model of trust it produced was built entirely on the best case. But a conduit only carries what you feed it. Where a patient has learned, from experience, to distrust the system that clinician represents, it delivers that distrust straight to the tool. Borrow trust and you borrow its absence too.

The assumption inside the model

That same study found the relational piece matters most in exactly the setting consumer health apps live in: long-term, low-urgency care. When care crawls along over months — a mood tracker, a chronic-condition companion, an ambient wellness app — the questions about relationships and governance move to the front of the line. That's the problem. The leg that carries the most load is the one built on a clinician who isn't there.

So the whole model has a fault line running through it. Its most fragile leg is load-bearing precisely where it's least available. And for the well-served patient inside a trusted relationship, the fault never shows. It shows only when you hand the tool to someone who never had reason to trust the system in the first place.

That's the person I want to design for. Not because it's noble, but because it's where the real test is.

When there's no one to vouch

You'd guess that trust in the AI collapses along with trust in the clinician. It's stranger than that. Another study — Black and white participants, sorted by how much they distrust the medical system — found that for low-mistrust people, the familiar pattern held: they trusted the human doctor more than the AI. For high-mistrust people, that gap closed. AI and human came out even.

This is not "marginalized patients prefer the machine." The gap didn't close because the AI rose. It closed because the human fell. The doctor's advantage, the whole thing the borrowed-trust model runs on, simply evaporated. And the Black participants with the highest mistrust rated both the human and the AI lowest of anyone. Not a vote for AI. A vote of no confidence in the whole arrangement.

This isn't uniform, most patients in this research did trust the system with their data; the distrust is concentrated, warranted, and specific, not a blanket trait.

So the reflex fix — "put a human in the loop; people trust humans" — is aimed at the wrong target. For the patient who trusts the human least, adding more human is adding more of the thing they don't trust. And when those same patients do ask for a human to check the AI's work (some of them do), they're not asking for a reassuring face. They're asking for someone who verifies the output and can be held responsible when it's wrong. Keep that human. Just stop asking them to be a security blanket and start asking them to be a safeguard.

Your app is the hard case, not the easy one

If the clinician can't anchor trust for the patient who distrusts the system, then it stops mattering whether there's a clinician in the loop at all. Present or absent, the relational route is closed for that person. Which flips how we're supposed to think about consumer health apps.

We build them for the easy user: the person who shows up in reasonably good faith, ready to be persuaded — trust as the default, skepticism as the edge case you'll handle later. But if the relational anchor was never going to hold for a large share of users anyway, that assumption isn't just optimistic; it's backwards. The skeptic isn't the edge case. The skeptic is the case that shows you what the product is actually worth because they're the ones who won't extend a cent of trust the thing hasn't earned. Design for the trusting user and you never find out. Design for the one who's been let down before, and everything you build has to stand on its own evidence, which is the only place trust was ever real to begin with.

So, the thing you build has no one to vouch for it. No relationship to inherit, no coat to borrow, and for a lot of users, a running start into the red. That sounds like the worst possible position to design from. I'd argue it's the most honest one. Every other setting lets the product hide behind someone else's credibility. Really it has to earn trust out loud — on its own evidence, in the open, with nothing on loan. That's not the constraint. That's the whole job.

Earning it out loud

We don't have to guess what that looks like. Ask high-mistrust patients what would make them trust a health AI, and they don't ask for warmth. They ask for evidence, and they're specific about which.

They want numbers, not adjectives: the actual accuracy, stated plainly, including how often it's wrong. They want to know it works for them. Black patients in that research asked pointedly whether the thing had been tested on skin like theirs, because they already know what happens to tools built on datasets that forgot they exist. They want to know it's been around, because tried-and-true beats brand-new when you're deciding whether to believe something with your health. They want to see the work — not a black box that emits a verdict, but a system that shows what it looked at, what it weighed, and why it landed where it did. And they want someone outside the building to vouch: a certification, an audit, a stamp that doesn't come from the people selling it.

Notice what every one of these has in common. Not one of them is a feeling. They're all checkable. That's the through-line: the trust these users will extend is the trust they can verify for themselves. Which means our job isn't to make the product feel trustworthy. It's to make it legible enough that a skeptic can confirm it is or catch it if it isn't.

Why warm and friendly backfires

Here's where I part ways with my own field. Consumer health design has a house style, and it's affection: the friendly mascot, the encouraging copy, the streak you don't want to break, the gentle line in the footer about how much we value your privacy and care about fairness. For most users, it works. It lowers the temperature, makes a clinical thing feel human, gets people through the door. For the users I've been describing, it does the opposite.

The research is almost uncomfortable to read on this point. Shown a warm reassurance that the AI was built to be "fair," high-mistrust patients didn't relax. Some trusted it less. One heard "fairness" and clocked it as the language of the very institutions that had failed her. The strongest anti-discrimination statement in the study got called "lip service" — fine words, no evidence, prove it. The warmth isn't neutral for these users. It's a tell. They've learned that soothing without substance is what you say right before you don't deliver. So the mascot isn't disarming; it's the thing standing where the evidence should be.

I'm not saying warmth is bad, or that friendly design is a mistake. For a huge swath of users it's exactly right, and I'll keep building it. I'm saying it's not load-bearing, and we've been treating it like it is. Be warm and be checkable. But when you can have the reader believe only one of them, make it the second.

The trust worth having

I started this thinking about a question I now think is the wrong one: how do we get skeptical people to trust the AI? Look at it straight and it's a little misguided. It treats trust as the goal and the person as the obstacle — something to be overcome, engineered around, converted. And for someone whose distrust is earned, who is skeptical because the system gave them every reason to be, succeeding at that question is the worst thing you could do. You'd have talked someone out of a caution that was protecting them. You'd have manufactured trust the product hadn't earned. That's not a win. That's the original harm, wearing a nicer interface.

So I'd throw the question out and replace it with a better one. Not "how do I get them to trust it," but "how do I make it trustworthy enough that they can see it for themselves, and how do I make sure they'll catch it when it slips." Engineered trust wants the user credulous. Earned trust wants them equipped. One asks them to lower their guard; the other hands them the tools to keep it up and check the product against it anyway. Only one of those survives contact with a person who's been burned before. And it means the skeptic's distrust isn't a conversion funnel. Sometimes it's the correct reading, and a product that respects that will occasionally have to accept a "not yet."

You can't borrow that. You can't put a face on it or warm it into being. You earn it out loud, in the open, on evidence a stranger can check. Or you don't get it. For the users who've been let down before, that's not the hard way to build trust.

It's the only honest one.

Sources

Duong T, Plage S, Woods L, et al. Consumer perspectives on trust in and benefits of artificial intelligence in health care. JAMA Netw Open. 2026;9(8):e2626916. doi:10.1001/jamanetworkopen.2026.26916. https://jamanetwork.com/journals/jamanetworkopen/fullarticle/2852422
Lee MK, Rich K. Who is included in human perceptions of AI? Trust and perceived fairness around healthcare AI and cultural mistrust. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI '21). Association for Computing Machinery; 2021. doi:10.1145/3411764.3445570. https://minlee.net/materials/Publication/2021-CHI-AIInclusion.pdf
Rinderknecht F-A, Yang VB, Tilahun M, Lester JC. Perspectives of Black, Latinx, Indigenous, and Asian communities on health data use and AI: cross-sectional survey study. J Med Internet Res. 2025;27:e50708. doi:10.2196/50708. https://www.jmir.org/2025/1/e50708/PDF