Why should data filtering be treated as part of the scientific method in clinical machine learning?

Filtering changes the prediction problem itself

Clinical machine learning studies usually begin with preprocessing: missing values, outliers, long gaps in patient records, irregular measurements, and physiologically implausible observations are removed or transformed before training. The paper's central claim is that these steps are not neutral. A filter that selectively discards or modifies observations based on their values will, by definition, alter the statistical properties of the output dataset.

That matters because the altered dataset may no longer represent the same prediction problem. Filtering can remove difficult cases, reduce variance, separate class distributions, or otherwise change the data in ways that improve apparent model performance without necessarily improving clinical usefulness. The authors frame this as a validity and comparability problem: researchers may compare algorithms on published datasets without fully understanding how preprocessing decisions shaped the data those algorithms were trained and tested on.

Eleven filters, two stages, one ICU cohort

The study examines eleven filtering strategies applied to raw vital-sign data, partitioned into two stages. Pre-aggregation filters operate on individual measurements within each list-valued hourly cell; post-aggregation filters operate on the matrix produced after hourly aggregation. Each filter is applied independently to the unfiltered dataset, producing a separate filtered dataset, and filters are not applied sequentially except where explicitly noted.

The pipeline is compatible with three publicly available ICU databases — MIMIC-III, MIMIC-IV, and eICU — but the work focuses primarily on MIMIC-III because it underlies the MIMIC-Extract pipeline used as the reference baseline. Each valid admission is represented as a multivariate time series spanning 24 hourly intervals and 7 vital-sign channels, giving a raw observation matrix of 24 × 7. Two binary tasks are defined per admission: Prolonged ICU Stay is positive when total ICU time across all episodes within the admission exceeded three days, and Mortality is positive when the patient died during the hospital stay.

Outlier bounds remove artifacts, not clinical events

Seven pre-aggregation filters remove physiologically implausible measurements from each hourly cell before aggregation, using inclusive valid ranges: heart rate [1, 600] bpm, systolic blood pressure [1, 400] mmHg, diastolic blood pressure [1, 300] mmHg, mean blood pressure [1, 300] mmHg, respiration rate [1, 70] breaths/min, temperature [21, 50] °C, and oxygen saturation [1, 100] %. Their union is evaluated as an All Vitals composite filter.

The authors are explicit that these bounds are not meant to remove valid data produced by a real event. During ventricular fibrillation, an adult heart rate can reach up to 500 beats per minute, and the heart-rate filter would preserve that value because it is physically possible and clinically relevant. A negative heart rate or one of several thousand beats per minute, by contrast, cannot occur in a human heart and is treated as a sensor error or artifact. Hourly cells whose lists become empty after filtering are set to NaN.

The paper reports how much this value-level filtering changes the data. In a heart-rate summary of MIMIC-III across four aggregation levels, the raw dataset contains 1,158,296 records with a maximum of 9,999,999 bpm and a standard deviation of 9,291.5; after applying the physiological plausibility range, 1,157,789 records remain with a maximum of 459 bpm and a standard deviation of 24.4. After mean aggregation over the first 24 hours before the heart-rate outlier filter, 45,985 records show a maximum of 555,652 and a standard deviation of 2,590.9; after the filter, 45,989 records show a maximum of 251.5 and a standard deviation of 25.5.

Record-level filters hurt mortality prediction

Three post-aggregation filters remove entire patient records rather than individual measurements. Long Missing Segment removes a record if any single vital-sign column contains a run of 10 or more consecutive missing or zero values. Long Gap removes a record if two or more consecutive hourly rows are empty, meaning all seven vital signs are simultaneously missing or zero. High Invalid Data removes a record if the fraction of invalid cells (NaN or zero) across the entire 24 × 7 matrix exceeds 10%, a threshold that retains records missing up to 16 of 168 possible cell values.

The authors argue these filters target records whose monitoring was intermittent, which in an ICU setting is likely correlated with severity of illness, transfer events, scans, or procedures. Removing such records shifts the sicker cohort toward patients with more complete and more regular vital-sign trajectories, effectively blurring the line between classes. In the mortality-prediction results under mean aggregation, all record-level filters show a negative impact: Long Missing Segment deviates by −0.0123 in test accuracy and −0.0069 in macro F1, Long Gap by −0.0359 and −0.0307, and High Invalid Data by −0.0305 and −0.0171, relative to the raw unfiltered baseline.

The authors note that the criteria for these filters could be loosened so that they affect fewer records, but they argue that any record-level filter would still create this problem in the data.

Imputation shifts nothing the centroid can see

The Fill Missing Data filter is different in character from both outlier removal and record gating. Applied post-aggregation, it imputes NaN cells in each vital-sign column using k-nearest-neighbour regression with k = 3 neighbours, operating independently along the time axis, and only when at least two known values exist in the column. The imputed value at a missing time index is the distance-weighted mean of the three temporally closest observed values. This reflects the observation that ICU documentation is episodic rather than continuous, so short gaps are more likely to represent charting intervals than true physiological absences.

The paper explains why this filter produces no visible centroid shift. The per-patient centroid used for the distribution-level comparison is computed by averaging only the unmasked cells of each 24-hour vital column. Cells that were NaN before filtering contributed nothing to the pre-filter centroid; after imputation they carry values close to the local empirical mean. Each patient's centroid therefore shifts by at most the imputation noise, and once aggregated across tens of thousands of records, the raw and filtered empirical cumulative distributions are visually indistinguishable for every vital.

The authors stress that this is not a deficiency of the filter but a structural property of the metric: the centroid summary is inherently blind to whether missing cells remain masked or are imputed near the local mean. Any signal imputation provides for downstream models will only emerge in metrics sensitive to within-record dispersion or per-vital data coverage.

High Invalid Data separates classes artificially

The paper's most decision-relevant comparison concerns whether observed performance improvements reflect a genuine increase in learning signal or merely an artificial simplification of the classification problem. The authors measure deviation in test accuracy and macro F1 relative to the raw unfiltered baseline, using a Random Forest classifier evaluated over four random seeds with five-fold cross-validation, supplemented by McNemar tests to assess whether the pattern of correct and incorrect predictions changes significantly.

For the High Invalid Data filter, the results show a slight positive shift in accuracy alongside a notable improvement in macro F1. The authors connect this to centroid behaviour: for heart rate, systolic blood pressure, diastolic blood pressure, mean blood pressure, and oxygen saturation, there is a consistent positive shift in the mortality-positive class and a consistent negative shift in the mortality-negative class. A shift pattern of sufficient magnitude means either the two classes are being made more similar or their separation is being increased. Because the High Invalid Data filter shows increased separation, the authors conclude it is effectively pulling the centroids further apart, artificially making the classification problem easier despite physically improbable shifts in the underlying vitals.

The paper also notes that the magnitude and direction of a centroid shift from a filter can differ between outcome groups, and that such a shift calls for careful examination because it may indicate the underlying ML task is being made artificially easier through a fundamental change in the dataset.

Why hourly flattening limits the claim

The study's primary limitation is its reliance on a flattened hourly aggregation architecture. While standard for benchmarks, hourly flattening smooths away sub-hourly physiological volatility and informative missingness (MNAR) patterns that the authors describe as deeply prognostic of acute patient deterioration. The authors state that future architectural phases should explore pipeline designs capable of preserving and evaluating raw physiological volatility and sampling dynamics directly.

This limitation qualifies how far the filtering findings transfer. The eleven filters were evaluated on hourly-aggregated 24 × 7 matrices derived from ICU admissions, with two binary tasks defined on MIMIC-III and the MIMIC-Extract pipeline as the reference baseline. The results characterize how filtering reshapes data distributions and downstream task difficulty at that aggregation level; they do not establish what happens when sub-hourly volatility and sampling dynamics are preserved.

What to report and what to watch

The paper's practical priority follows from its own comparison: the filters that most clearly changed task difficulty were not the value-level outlier bounds, which removed physiologically impossible measurements while preserving clinically relevant extremes, but the record-level filters that removed entire patients and the High Invalid Data filter that increased class separation. The authors argue that providing code or cutoff tables alone is not enough; researchers also need to explain why each filtering decision is appropriate for the clinical and biological problem being studied if the result is intended for application.

For readers evaluating clinical ML results, the paper's specific warning is that a positive accuracy or macro F1 deviation relative to a raw baseline is not by itself evidence of a better model. The relevant check is whether the filter changed the class structure — for example, by shifting mortality-positive and mortality-negative centroids in opposite directions across heart rate, blood pressure, respiration rate, temperature, or oxygen saturation — and whether that shift corresponds to a genuinely easier problem rather than a better solution to the same one. The authors' broader claim is that filtering choices should be explicitly reported, justified, and analyzed as an integral part of the experimental design, because the same filtering rule can help one task and hurt another.

Sources used in this answer