How accurate are AI pathology models overall?
A large meta-analysis of 100 studies covering over 152,000 whole slide images found that AI models achieve a mean sensitivity of 96.3% and specificity of 93.3% [1]. Sensitivity means the model correctly identifies 96 out of 100 actual disease cases; specificity means it correctly rules out disease in 93 out of 100 healthy cases. These numbers suggest that, on average, AI performs at a level comparable to or better than human pathologists for the specific tasks studied. However, the same review noted that 99% of the included studies had at least one area of high or unclear risk of bias, meaning the real-world accuracy for diverse populations could be lower than these headline numbers suggest [1].
Foundation models vs. task-specific training: which handles diversity better?
The largest validation study in this set—covering over 100,000 prostate biopsies from 7,342 patients across 15 sites in 11 countries—directly compared two types of AI: foundation models (large, general-purpose models pre-trained on vast data) and task-specific models (trained from scratch on the target task) [5]. The key finding: foundation models did not universally outperform task-specific models. When enough labeled training data was available, task-specific models matched or even beat foundation models, and they produced fewer clinically significant misgradings [5]. Foundation models did show an advantage when training data was scarce—a common scenario for rare diseases or underrepresented populations—but they also used up to 35 times more energy, raising sustainability concerns [5]. For clinical use, the study argues that rigorous validation on the target population is more important than choosing a foundation model.
What about the latest AI assistants like PathChat?
A 2024 study introduced PathChat, a vision-language AI assistant trained on over 456,000 pathology-specific visual-language instructions [4]. It outperformed GPT-4V (the model behind ChatGPT-4) on multiple-choice diagnostic questions across diverse tissue types and diseases, and human pathologists preferred its responses [4]. While this suggests that specialized AI can handle a wide range of pathology tasks, the study did not specifically test performance across different patient demographics (e.g., race, ethnicity, or socioeconomic status). So while PathChat is a promising tool for education and research, its ability to handle diverse patient populations remains unexamined.
About These Sources
This answer is built on 5 peer-reviewed studies — published from 2022 to 2025, 4 from 2024 or later, 3 in Q1 journals, collectively cited 497 times — selected as the most relevant from 5 studies that passed quality screening, drawn from 65 papers retrieved from a database of over 500 million.
Sources used in this answer
Artificial intelligence in digital pathology: a systematic review and meta-analysis of diagnostic test accuracy
In a systematic review and meta-analysis of 100 studies (48 in meta-analysis) covering over 152,000 whole slide images, AI models showed a mean sensitivity of 96.3% and specificity of 93.3%, but 99% of studies had at least one area of high or unclear risk of bias, indicating that reported accuracy may not reflect real-world diverse populations.
Confounders mediate AI prediction of demographics in medical imaging
Using over 530,000 cardiac ultrasound videos from two healthcare systems, this study found that AI models could predict sex (AUC 0.85) and age (mean error 9.12 years) but could not reliably predict race unless the training data was manipulated to include confounding variables like age and sex, showing that AI can learn demographic shortcuts rather than true disease features.
Melanoma in Chile: demographics and clinico-pathological features.
A multicenter cohort study of 1,037 melanoma patients in Chile found that 9.3% had acral lentiginous melanoma—a subtype rare in lighter-skinned populations—and that survival disparities existed between public and private healthcare settings, highlighting the need for population-specific data to train and validate AI models.
A multimodal generative AI copilot for human pathology
PathChat, a vision-language AI assistant trained on over 456,000 pathology-specific instructions, outperformed GPT-4V on multiple-choice diagnostic questions across diverse tissue types and was preferred by human pathologists, but its performance across different patient demographics was not specifically tested.
Foundation Models -- A Panacea for Artificial Intelligence in Pathology?
In the largest validation of AI for prostate cancer diagnosis—over 100,000 biopsies from 7,342 patients across 15 sites in 11 countries—foundation models did not universally outperform task-specific models; task-specific models matched or surpassed them when sufficient labeled data was available, and foundation models used up to 35 times more energy.