Can machine learning classify digital exclusion in ageing populations across cohorts?

A narrow outcome standing in for a broad idea

Digital exclusion is a multidimensional construct: it can mean missing connectivity or devices, unaffordable access, low skills or confidence, accessibility barriers, dependence on others, or difficulty with specific tasks. None of these dimensions is measured consistently across the five surveys the study uses, so the authors chose the one indicator available in all of them. Participants who reported using neither the internet nor email at the survey wave were classified as digitally excluded; anyone reporting either was counted as digitally participating.

That choice sets the boundary of everything that follows. The measure captures basic non-participation, not digital capability, autonomy, frequency or breadth of use, or the ability to complete a task such as navigating a patient portal. A respondent who reports internet use may still be unable to use telehealth independently, and a non-user may rely effectively on assisted access. The paper is explicit that its binary outcome is a harmonised proxy, and the results should be read at that level.

Prevalence ranges from 22.5% to 86.2% across cohorts

The pooled analytic sample contained 71,689 middle-aged and older adults: 17,969 in CHARLS (China), 7,916 in ELSA (England), 16,439 in HRS (USA), 15,419 in MHAS (Mexico), and 13,946 in SHARE (a multinational European cohort treated as one pooled cohort in the main modelling). Reported non-use of internet and email ranged from 22.5% in ELSA to 86.2% in CHARLS, with HRS at 38.9%, MHAS at 52.5% and SHARE at 54.6%.

Two gradients recurred most clearly. Exclusion rose steeply with age — in CHARLS it reached 98.9% among those aged 80 and over, and in SHARE 92.5% — and it was consistently concentrated among participants with lower educational attainment. Sex differences were smaller and did not point the same way everywhere: prevalence was higher among women in CHARLS, ELSA and SHARE, while HRS and MHAS showed little or no female excess.

SHARE's pooled 54.6% hides a wide spread. In a separate country-identifiable Wave 7 extract used only for description, exclusion ranged from 20.1% in Denmark to 79.2% in Poland, with Spain at 78.6%, Italy at 75.4% and Greece at 74.7%. These are unweighted sample estimates from an ancillary analysis that was not used for model fitting, tuning, calibration or validation, so they should not be read as nationally representative prevalence.

Fourteen routine variables, tested on held-out people

The candidate features were fixed in advance rather than screened automatically, to keep the five cohorts comparable. The set comprised 14 variables: age group, sex, marital status, education level, household wealth quartile, household income quartile, employment status, smoking, alcohol consumption, physical activity, self-rated health, limitations in activities of daily living, limitations in instrumental activities of daily living, and chronic disease count. Internet use was excluded because it defines the outcome and would leak the label; depression was excluded from the features and analysed separately.

Six classifiers were compared within each cohort: logistic regression as a transparent linear benchmark, plus random forest, gradient boosting, AdaBoost, XGBoost and LightGBM to test whether nonlinearities and interactions added anything. Models were trained with stratified 5-fold cross-validation, and performance was measured on out-of-fold observations — people not used to fit the fold that scored them. Preprocessing, imputation, tuning and calibration were confined to training folds. Discrimination was summarised by AUC, the probability that a randomly chosen excluded person receives a higher score than a randomly chosen participating person; because AUC can look favourable when one class dominates, the authors also report precision-recall AUC and Brier scores.

Gradient boosting wins, but only by 0.001 to 0.013 AUC

Gradient boosting achieved the highest AUC in all five cohorts: 0.821 in CHARLS, 0.796 in ELSA, 0.839 in HRS, 0.734 in MHAS and 0.850 in SHARE. Logistic regression reached 0.813, 0.795, 0.833, 0.721 and 0.846 respectively. The gap between the flexible model and the linear benchmark therefore ranged from 0.001 to 0.013 AUC.

That comparison is the paper's central methodological result. Much of the discriminative signal can be represented by stable additive gradients, so greater algorithmic complexity did not buy materially better separation in these data. The authors frame the machine-learning exercise as a test of whether nonlinearity and interactions add practically meaningful discrimination, not as a search for the best algorithm.

Performance also varied by cohort in ways that belong to the population as much as to the model. Discrimination was highest in SHARE and HRS and lowest in MHAS, which the authors attribute to plausible differences in case mix, prevalence, measurement and the strength of feature-outcome relationships. Precision-recall AUC ranged from 0.496 in ELSA to 0.962 in CHARLS, tracking cohort prevalence as expected, and gradient boosting Brier scores ranged from 0.095 in CHARLS to 0.206 in MHAS. Reading discrimination and calibration together gives a fuller picture than AUC alone.

Education, income, age and wealth recur, but their order shifts

Explanation analyses of the fitted models clustered contributions around a coherent set of social, economic and health characteristics. Education, income quartile, age group, wealth quartile and alcohol consumption were recurrent contributors, while sex, employment, functional limitations, self-rated health and chronic disease count varied more by cohort. Chronic disease burden and functional limitation were especially prominent in MHAS.

The important point is what these quantities mean. They describe how the models used observed features to classify concurrent status; they are not causal effects, and they do not estimate how intervening on a feature would change the outcome. The cross-sectional design cannot establish whether poorer health precedes non-use or whether both reflect common underlying disadvantage. The recurrence of the same broad domains alongside substantial variation in their relative importance is precisely the rationale for testing transportability: a shared feature set does not imply shared feature-outcome relationships.

Transfer to an unseen cohort costs about 0.026 AUC

Leave-one-cohort-out validation trained a logistic regression on four cohorts and tested it on the fifth, then compared that external AUC with the cohort's internal AUC. Every held-out cohort retained above-chance discrimination, but performance fell in all five, with a mean AUC attenuation of 0.026. The smallest gaps appeared in CHARLS (0.813 internal versus 0.799 external, a gap of 0.014) and ELSA (0.795 versus 0.775, 0.019), followed by HRS (0.833 versus 0.810, 0.023) and SHARE (0.846 versus 0.818, 0.029). MHAS showed the largest decline (0.721 versus 0.675, 0.046).

This is partial transportability, not universal applicability. The authors note that transfer was stronger where the held-out cohort resembled the pooled derivation data and weaker where feature distributions and relationships differed, but the study was not designed to isolate which contextual factor produced each gap, so those explanations remain plausible interpretations rather than established mechanisms. Intercept recalibration improved probability calibration where baseline prevalence shifted, but it cannot repair changes in the underlying feature-outcome relationships. The practical consequence is that local validation remains necessary even when a shared signal clearly exists.

Higher scores track concurrent depression, not future depression

As a separate health-relevance check, the authors took cross-fitted classification scores from the internal 5-fold procedure and related them to depression measured in the same wave, adjusting for age, sex, education level and chronic disease count. A one-standard-deviation higher score was associated with greater odds of concurrent depression in all cohorts, with a pooled adjusted odds ratio of 2.23 (95% CI 1.83–2.71). Depression was more common among digitally excluded than digitally participating participants in all five cohorts.

Because the score, the features and depression were all measured at the same wave, temporal ordering is unresolved and residual confounding is possible. The authors read the result as evidence that the classification score captures a broader pattern of contemporaneous vulnerability — not as evidence that digital exclusion causes depression, and not as a prediction of future mental-health outcomes.

Population description, not individual screening

The paper's own conclusion is deliberately restrained: digital exclusion has a reproducible multivariable structure, but these models are descriptive classifiers of concurrent status rather than prognostic tools. Three findings carry that conclusion. The advantage of gradient boosting over logistic regression was at most 0.013 AUC, which weakens the case for algorithmic complexity unless future prospective work shows clearer gains in discrimination, calibration, decision utility or equity. Cross-cohort transfer lost a mean 0.026 AUC and required recalibration. And the outcome itself captures only basic non-participation, not the skills, affordability, accessibility or autonomy dimensions that determine whether someone can actually use a digital health service.

For policy and service design, the results argue against a single universal screening rule. Population-level characterisation can identify where basic non-participation concentrates — lower education, older age groups, and settings such as Poland, Spain, Italy and Greece within SHARE — but the priority behind that concentration may differ, with connectivity and affordability dominating in one context and skills, accessibility or assisted service design in another. The models were not validated for denying, restricting or automating access to services, and the authors stress preserving telephone, in-person and assisted routes. The most defensible use of this work is therefore to compare settings and groups, and to justify local validation before any model is applied to individuals.

Sources used in this answer