How well does multimodal clinical AI systems handle diverse patient populations?

Does combining data types actually improve accuracy for diverse patients?

Yes, but the improvement is not automatic—it depends on how the data is combined and whether the model has seen diverse examples. The strongest single finding comes from a study of GPT-4V on 93 complex clinical cases: multimodal inputs (text + image) achieved 80.6% diagnostic accuracy, compared to 66.7% with text alone and 45.2% with images alone [1]. This pattern holds across specialties: a scoping review of 432 papers found that multimodal AI outperformed unimodal models by an average of 6.2 percentage points in AUC [5]. For prostate cancer detection, combining MRI-based deep learning with clinical parameters (PSA, age, prostate volume) raised AUC from 0.70 (MRI alone) to 0.77 [3]. However, these gains only materialize when the model has been trained on data that reflects the target population—and that is where many systems fall short.

How well do these systems work on patients who are not white, male, or young?

The evidence shows a troubling gap: performance degrades significantly for patients with darker skin tones, uncommon diseases, and non-majority demographic groups. In dermatology, state-of-the-art AI models tested on the Diverse Dermatology Images dataset—the first publicly available dataset with pathologically confirmed images across diverse skin tones—saw their ROC-AUC drop by 27–36% compared to original test results, with all models performing worse on dark skin tones and rare diseases [8]. A large-scale evaluation of diabetic retinopathy screening algorithms across 202,886 encounters (32% white, 17% Black, 39% South Asian) found that only 8 of 25 eligible vendors agreed to participate, and processing times ranged from 13.5 hours to 105 days, raising practical equity concerns [2]. The problem is not just skin deep: a fairness analysis across dermatology, radiology, and ophthalmology found that indiscriminately including all patient metadata (age, gender, skin tone) can actually harm fairness for protected subgroups, and that careful metadata selection is essential [6]. A separate legal and ethics review warns that failing to account for gender and sex differences in medical AI risks misdiagnosis and discrimination, especially for marginalized communities [7].

Can these biases be fixed, and what are the practical limits?

Yes, but the fixes require deliberate effort and transparency. The dermatology study found that fine-tuning AI models on the diverse DDI dataset closed the performance gap between light and dark skin tones, and the fine-tuned models actually outperformed dermatologists on identifying malignancy in dark skin [8]. Similarly, a multimodal framework for cardiovascular screening in athletes used structured sparse inference and exertion-conditioned embeddings to improve risk stratification accuracy across diverse athletic populations [4]. However, the path to fairness is not straightforward: the fairness guideline study showed that simply adding more data modalities can backfire if metadata is not carefully selected—the wrong metadata can worsen bias [6]. A major practical barrier is that many commercial AI systems are proprietary, making it impossible to audit their training data for biases [1]. The largest real-world evaluation here—the diabetic retinopathy screening study—took 96 to 460 days just to install and verify algorithms, and only a third of eligible vendors participated [2]. This suggests that even when solutions exist, deployment at scale remains slow and uneven.

About These Sources

This answer is built on 8 peer-reviewed studies — published from 2022 to 2025, 5 from 2024 or later, 3 in Q1 journals, collectively cited 143 times — selected as the most relevant from 10 studies that passed quality screening, drawn from 49 papers retrieved from a database of over 500 million.

Sources used in this answer

Related research questions