How well does medical foundation models handle diverse patient populations?

How well do these models adapt to different hospitals and patient groups?

A large multi-center study tested a publicly available EHR foundation model (trained on 2.57 million records from one US hospital) on data from two other hospitals: a Canadian children's hospital (SickKids) and the MIMIC-IV intensive care database. The model adapted remarkably well: after continued training on just 1% of the local data, it matched the performance of a traditional model (gradient boosting machine) trained on all the local data, and it was 60–90% more sample-efficient than training a new foundation model from scratch [1]. This means that even a model trained on a relatively homogeneous population can be efficiently tuned to work well in a very different setting, which is good news for deployment in diverse clinical environments.

However, the same study also found that the off-the-shelf model (without any local tuning) only matched the traditional model's performance when given all the data, and its main advantage was in low-data scenarios — a 13% improvement when training labels were scarce [1]. So the model's ability to handle diversity is real but conditional: it requires at least a small amount of local data to adapt, and it shines most where data is limited, not where abundant local data already exists.

What about rare diseases, non-English languages, and safety?

A comprehensive evaluation of 11 foundation models (7 general-purpose, 4 medical-specialized) across seven medical tasks found that general-purpose models actually outperformed medical-specialized ones, with a median hallucination-free response rate of 76.6% vs. 51.3% [2]. This is a critical finding: models explicitly trained on medical data were more likely to produce factually incorrect or clinically dangerous outputs. Physician audits revealed that 64–72% of these errors came from faulty causal or temporal reasoning, not from missing knowledge [2]. For diverse patient populations — where rare diseases, atypical presentations, or non-English languages are common — this reasoning failure is especially dangerous because the model may confidently generate a plausible-sounding but wrong answer.

A separate study directly addressed the problem of rare diseases and non-English languages by developing a multimodal, multidomain, multilingual foundation model. It achieved strong performance on nine datasets covering 2 infectious and 14 non-infectious diseases, including rare conditions where labeled data is scarce or absent [6]. This suggests that training on diverse data types (images, text) and languages can help, but the model still relies on the breadth of its training data — it's not a magic bullet.

A global survey of 70 clinicians found that 91.8% had encountered medical hallucinations from AI, and 84.7% believed they could cause patient harm [2]. So while foundation models can be adapted to diverse populations, their tendency to hallucinate — especially on complex or unusual cases — remains a serious safety concern that limits their real-world use.

What training strategies lead to better performance across diverse groups?

The strongest evidence points to training on large, diverse, multimodal datasets. A dermatology foundation model (PanDerm) was pretrained on over 2 million real-world skin disease images from 11 institutions across 4 imaging modalities. It achieved state-of-the-art results on 28 benchmarks, often outperforming existing models using only 10% of labeled data [3]. In reader studies, it improved clinicians' diagnostic accuracy by 11% on dermoscopy and boosted non-dermatologists' differential diagnosis by 16.5% across 128 skin conditions [3]. This shows that breadth of training data directly translates to better handling of diverse patient presentations.

Similarly, a generalist vision-language model (BiomedGPT) trained on diverse biomedical data achieved state-of-the-art results in 16 out of 25 experiments, with a low error rate of 3.8% in question answering and 8.3% in writing radiology reports [4]. The authors explicitly note that effective training with diverse data leads to more practical AI for improving diagnosis and workflow efficiency [4].

On the other hand, a study on cancer-specific foundation models found that most current single-cell models are trained predominantly on healthy cells, with a strong bias toward peripheral blood mononuclear cells, raising questions about their generalization to cancer contexts [5]. This suggests that narrow training data — even if it's large — can limit a model's ability to handle diverse patient populations, especially those with disease-specific molecular features.

About These Sources

This answer is built on 6 peer-reviewed studies — published from 2024 to 2025, 6 from 2024 or later, 5 in Q1 journals, collectively cited 322 times — selected as the most relevant from 6 studies that passed quality screening, drawn from 56 papers retrieved from a database of over 500 million.

Sources used in this answer

Related research questions