Can early ADAS-Cog response predict long-term cognitive change after rTMS?

Two summaries of the same early response

The paper asks a narrow prognostic question: once a participant has been observed through Week 5 of an rTMS trial, does the early change in the ADAS-Cog total score help forecast that participant's later ADAS-Cog change? And does splitting that same early change into individual cognitive components add anything? The ADAS-Cog total here is an 11-component score ranging from 0 to 70, with higher values meaning greater impairment; the components cover recall, naming, commands, constructional and ideational praxis, orientation, word recognition, language, comprehension, word finding, and remembering test instructions.

The comparison is deliberately matched. One model represents early response as two numbers, the Week-3 and Week-5 total-score changes from baseline. The alternative replaces those two numbers with 22 variables, the Week-3 and Week-5 changes on each of the 11 components. Both use the same participants, outcomes, cross-validation splits, preprocessing, and ridge tuning, so the difference in performance is attributable to how early response is represented rather than to a different modelling pipeline.

The authors frame this as a prognostic task, not a mechanistic one. They are not testing whether rTMS works; they are testing whether measurements already collected during the early treatment period carry information about what comes next.

Who entered the landmark cohort

The analysis is a secondary use of data from a randomized, double-blind, sham-controlled rTMS trial in Alzheimer disease that included 10 active sessions over 2 weeks, 20 active sessions over 4 weeks, and 4 weeks of sham. Of 112 randomized participants, 101 had complete predictor information through the Week-5 landmark. Future outcome data were available for 99 of those at 8 weeks post-treatment, 95 at 16 weeks, and 94 at 24 weeks; 91 participants had complete outcomes at all three horizons and formed the primary cohort.

That complete-case restriction matters for reading every number below. The authors describe participants included in the three-horizon cohort against those excluded for incomplete predictors or outcomes, and note that missingness may have affected cohort composition. All model development and validation happened inside this single two-site dataset, so the results describe internal reproducibility, not transportability to other clinics, protocols, or patient groups.

The prediction landmark is Week 5. Outcomes are future ADAS-Cog total change from baseline at 8, 16, and 24 weeks post-treatment, so negative values mean improvement relative to baseline and positive values mean worsening.

Baseline information alone predicted poorly

The predictor-set comparison is a hierarchy. The baseline-only ridge model used age, sex, and baseline ADAS-Cog total. The baseline-plus-trial model added treatment group and site. The early total-response model added the Week-3 and Week-5 total-score changes, giving seven predictors before categorical encoding. The early domain-response model kept the same baseline and trial covariates but swapped the two total-score changes for the 11 Week-3 and 11 Week-5 component changes, giving 27 predictors before encoding.

Baseline demographic and cognitive information alone provided little out-of-sample predictive value: the paper reports negative overall R² for both the baseline-only and baseline-plus-trial ridge models. Adding the two early total-score changes moved overall mean absolute error to 3.067, root mean squared error to 3.885, and R² to 0.384. In other words, the predictive signal came from what happened during the early treatment period, not from who the participants were at baseline.

These are out-of-fold errors averaged across the three horizons, produced by repeated treatment-site-stratified 5-fold cross-validation with five repetitions and nested hyperparameter tuning. They are not training-set fits.

Twenty-two component changes predicted worse than two totals

The central result is a comparison in the opposite direction from what a domain-specific hypothesis would predict. Replacing the two total-response variables with 22 component-level changes worsened performance: overall MAE rose to 3.363, RMSE to 4.269, and R² fell to 0.257. The paired participant-level bootstrap, with 10,000 resamples, put the MAE difference at −0.297 points in favour of the simpler total-response model, with a 95% confidence interval of −0.485 to −0.115.

The authors offer an interpretation rather than a demonstrated mechanism. Aggregating component scores may reduce measurement noise, while expanding two predictors into 22 substantially raises dimensionality relative to a sample of 91 participants. Regularization can reduce but not eliminate the instability of fitting many predictors in a small sample. The paper is explicit that this is not evidence that individual ADAS-Cog domains are clinically unimportant; it is evidence that decomposition did not help this particular prediction task.

Two sensitivity analyses support the main finding: using absolute future ADAS-Cog scores instead of change-from-baseline outcomes, and excluding Week-3 response information. The Week-3 exclusion is relevant because Week 3 fell after treatment completion for the 2-week arm but during the intervention period for the 4-week active and sham arms, so Week-3 response means different things across groups.

A mixed-effects model came close but not clearly ahead

A separate comparison asked whether the modelling approach matters when richer domain information is supplied. Full-domain ridge, multi-task elastic net, and a restricted random forest all received age, the 11 baseline components, the 11 Week-3 component changes, the 11 Week-5 component changes, sex, treatment group, and site. The longitudinal mixed-effects model instead used grouped memory, language, and praxis composites with a participant-specific random intercept, so this is a comparison of modelling strategies and predictor representations, not a pure algorithm contest on an identical design matrix.

Among the domain-rich specifications, the mixed-effects model had the lowest numerical overall error, with MAE 3.162 and R² = 0.318, followed by multi-task elastic net, full-domain ridge, and restricted random forest. But across all candidate models, the simpler early total-response ridge still had the lowest numerical overall MAE, 3.067, and the highest R², 0.384. The paired difference between the early total-response ridge and the mixed-effects model was small and uncertain: ΔMAE = −0.095, 95% CI −0.221 to 0.025.

The horizon pattern is not uniform. The early total-response ridge had lower error at 8 weeks, the mixed-effects model was slightly better numerically at 16 weeks, and the two were close at 24 weeks. The paper therefore does not claim that either approach is definitively superior. The near-equivalence of full-domain ridge and multi-task elastic net likewise suggests that shared sparsity across horizons added little over ordinary shrinkage in this sample.

Calibration and coefficient stability remain exploratory

For the early total-response ridge, calibration was assessed at each horizon by regressing observed ADAS-Cog change on the mean out-of-fold prediction across the five repetitions, with ideal calibration corresponding to an intercept of zero and a slope of one. This tells readers whether the model's predicted magnitudes track observed magnitudes, which is a different question from ranking participants correctly.

Coefficient summaries for the full-domain ridge model were treated as exploratory conditional predictive associations, not causal effects or evidence of independent biological importance. Sign consistency was summarized across the 25 outer fits, with a descriptive threshold of 0.80 used to flag relatively stable directions. Because the predictors are correlated and regularized, these coefficients should not be read as separable domain contributions.

The three future outcomes were only moderately correlated, with Spearman correlations of 0.463 between 8 and 16 weeks, 0.468 between 8 and 24 weeks, and 0.542 between 16 and 24 weeks. The horizons share information but are not interchangeable, which is why the paper reports per-horizon behaviour alongside overall averages.

An updating tool, not a treatment-selection rule

The practical reading is a priority ordering rather than a deployment recommendation. If the goal is to forecast later ADAS-Cog change for someone already observed through Week 5, the two-number total-score representation was the better-performing and simpler option in this sample, and the 22-component representation was not worth its added dimensionality. If the goal is to decide who should receive rTMS before treatment starts, this study does not address it: baseline information alone had negative overall R², and the model is explicitly described as an internally validated early-response updating tool rather than a pretreatment selection model.

A related distinction concerns thresholds. A decline of roughly three ADAS-Cog points has been proposed as clinically relevant in early Alzheimer disease, but the paper stresses that such a threshold is conceptually different from a prediction error and should not be treated as a formal cutoff for acceptable model performance. The reported MAE of about 3.067 points is an average absolute error across three horizons, not a guarantee that individual predictions fall within a clinically meaningful band.

The most useful conclusion is therefore conditional on timing and representation. Early aggregate ADAS-Cog response carried substantial prognostic information in this cohort, domain-level decomposition did not improve out-of-sample prediction, and a structured longitudinal model performed competitively without clearly beating the simple regularized model. Larger samples, independent external validation, and prospective evaluation are needed before any clinical use, and the study provides no evidence about causal rTMS efficacy.

Sources used in this answer