How accurate is AI for data-driven ADHD diagnosis across studies?

Two pooled numbers describe a very uneven literature

The paper asks how well data-driven models diagnose ADHD, and its headline answer comes from a random-effects meta-analysis of 54 eligible studies. Pooled sensitivity was 0.87 (95% CI 0.83–0.91) and pooled specificity was 0.91 (95% CI 0.88–0.93). In plain terms, the average study-level model caught about 87% of ADHD cases and correctly cleared about 91% of typically developing participants, but those are averages across studies that differ in cohorts, inputs, and validation design.

The heterogeneity is the part that changes how the numbers should be read. The forest plots report I² = 97.4% for sensitivity and 98.2% for specificity, and the authors note that residual heterogeneity remained above 96% in meta-regression. A pooled estimate this unstable describes the literature rather than predicting what a new model would do.

Publication bias was detected for sensitivity (Egger's test p < 0.001) but not for specificity, where the result was marginal (p = 0.052). That asymmetry matters because it means the sensitivity average may be shaped by which studies were published, while the specificity average carries a different, weaker concern.

Five assessment categories split the evidence

The 54 studies were sorted by the kind of data fed to the model: physiological indicator detection (6 studies), neuroimaging assessment (26), sensory and motor function assessment (9), behavioral and subjective reporting assessment (3), and multimodal fusion assessment (10). This grouping is the paper's main comparison axis, and it is also where readers can most easily overreach.

Category-level pooled sensitivity ranged from 0.79 for multimodal fusion (95% CI 0.70–0.85) to 0.90 for both neuroimaging (95% CI 0.82–0.94) and sensory and motor function assessment (95% CI 0.75–0.96). Physiological indicators pooled at 0.89 (95% CI 0.79–0.94), and behavioral and subjective reporting pooled at 0.87 with a very wide interval (95% CI 0.03–1.00).

The between-category tests were not statistically significant for sensitivity (p = 0.068) or specificity (p = 0.743). The authors therefore treat the ranking as descriptive, not comparative: a lower point estimate for multimodal fusion does not establish that fusion is worse, especially since some multimodal studies outperformed some unimodal ones.

Removing high-risk studies barely moved the averages

Risk of bias was assessed with PROBAST, a structured tool covering participants, predictors, outcomes, and analysis. Eleven studies (20.4%) were rated high risk overall, 11 (20.4%) unclear, and 32 (59.3%) low. The analysis domain produced all 11 high-risk judgments, and seven studies were also high risk in participant selection; no study was high risk in the predictor or outcome domains.

The natural test is whether those flawed studies drove the headline result. Excluding the 11 high-risk studies left pooled sensitivity at 0.87 (95% CI 0.82–0.91) and specificity at 0.92 (95% CI 0.88–0.94), both within 0.01 of the primary estimates. Heterogeneity, however, stayed at I² = 97.9% for sensitivity and 98.5% for specificity.

That combination is the paper's sharpest methodological point: high-risk studies are not the explanation for the spread. The authors attribute the remaining variance to unmeasured differences in feature engineering, tuning, acquisition protocols, comorbidity, and case definition, which means cleaning up study quality alone would not make these estimates transportable.

Independent test sets showed lower sensitivity

Most included models were evaluated retrospectively with internal cross-validation, often in small cohorts. Sample sizes across the literature ranged from 10 to 45,779, with a median of 121, so the review split studies at that median and also ran a meta-regression treating sample size as continuous.

The meta-regression indicated lower sensitivity with independent test sets than with cross-validation. The authors are careful about what this means: as an observational moderator analysis, it does not by itself prove that cross-validation inflates performance. It does show that the validation design is entangled with the reported numbers, so a reader comparing two studies should check whether either used a genuinely held-out sample.

Only 8 of the 54 studies evaluated adults, which limits how far any of these pooled figures extend across the lifespan. The pediatric weighting is a scope condition on the whole review, not a detail of one subgroup.

No modality or algorithm pairing can be ranked

Individual studies invite strong claims. Some EEG studies reported very high internally validated accuracy, and one wearable heart-rate-variability study reached 0.991 accuracy, while a retinal fundus study using XGBoost reached 0.969. But these are study-specific estimates from different cohorts and validation procedures, and the review did not pool them into a formal EEG-versus-fMRI comparison.

CNNs and SVMs were the most frequently used algorithm families, with CNNs learning hierarchical representations from high-dimensional inputs and SVMs applied to hand-engineered features in small-sample settings. The paper explicitly declines to conclude that any algorithm-modality pairing is superior, because algorithm and modality effects cannot be separated from differences in sample size, preprocessing, feature selection, and validation.

Multimodal fusion illustrates the same trap. Its pooled sensitivity point estimate was the lowest of the five categories, yet the between-category test was not significant, and the authors list heterogeneous fusion strategies, feature redundancy, scale imbalance, and small samples as plausible contributors. Missing modalities add a deployment problem: an MRI, EEG, or questionnaire stream may be absent because of cost, contraindications, or incomplete visits, and complete-case analysis can then shrink the effective sample and introduce selection bias.

External validation comes before clinical use

The paper's own framing is that current models identify candidate signals for ADHD classification, not that they are ready for care. Most were evaluated retrospectively and with internal cross-validation, so external validation across sites, scanners, populations, and clinical workflows is a prerequisite before any prospective assessment.

The authors propose a sequence rather than a single fix. Prospective shadow-mode evaluation would test performance in the background of active workflows without letting the model influence decisions. Reporting should follow TRIPOD+AI, covering data sources, participant flow, missing-data handling, preprocessing, model development, validation, discrimination, calibration, uncertainty, and subgroup performance. Early clinical evaluations should follow DECIDE-AI by prespecifying intended use, workflow, human-AI interaction, safety outcomes, and failure analysis.

Practical barriers sit alongside these methodological ones. The black-box character of many deep learning architectures restricts clinician trust, which is why the authors call for explainable AI that maps model weights back to biologically plausible ADHD features. Cost-effectiveness also remains unevaluated, and acquiring high-quality multimodal data demands substantial financial and temporal resources.

Standardize evaluation before ranking models

The review's conclusion is a priority ordering, not a verdict on any model family. Because between-category differences were not significant and heterogeneity stayed above 96% even after excluding high-risk studies, the authors argue that standardized reporting, common evaluation protocols, and independent multicenter validation should take precedence over further ranking of heterogeneous study-level estimates.

For a reader deciding what to take from this paper, the useful split is between the pooled figures and the conditions attached to them. The 0.87 sensitivity and 0.91 specificity are descriptive summaries of 54 studies, with publication bias detected on the sensitivity side and only 8 studies covering adults. The actionable claim is narrower and firmer: the field's next gain comes from making studies comparable and externally validated, not from declaring one modality or algorithm the winner.

Sources used in this answer