Can randomized probability perturbation detect poisoned samples under dataset imbalance?

Imbalance raises attack success and weakens defenses

The paper starts from a condition most prior-training defenses assume away: balanced training data. In its investigation, backdoor attacks succeed more often as the dataset becomes more imbalanced, and existing defenses degrade as imbalance grows. The authors attribute the first effect to a majority-class bias that makes the model easier to steer toward a dominant label.

The concrete comparison is a Badnets attack on MNIST with the same 0.3% poisoning ratio. Attack success rate rises from 13.8% at imbalance ratio ρ = 1 to 21.8% at ρ = 200. The pattern holds across the other attacks tested: WaNet goes from 35.8% to 55.1%, ISSBA from 30.2% to 53.3%, and Sleeper Agent from 28.6% to 48.6% over the same range. Clean-data AUC stays near 100 in these rows, so the attacks are not simply degrading the model's ordinary accuracy.

The defense side shows the same direction. Certified accuracy for RAB against Badnets falls from 41.5% at ρ = 1 to 15.7% at ρ = 200, and DPA falls from 56.2% to 14.8%. These are certified-accuracy percentages for two specific certified defenses, not a survey of all defenses, but they establish the gap the paper targets.

RPP measures how little a prediction moves under noise

RPP stands for Randomized Probability Perturbation. For a given input, the method records the model's output probability vector, adds isotropic Gaussian noise to the input J times, and averages the largest change in that vector across the noisy copies. The score is therefore a per-sample stability measure computed only from black-box output probabilities; no gradients or internal weights are needed.

The key empirical observation is that poisoned inputs produce consistently lower RPP scores than clean ones across imbalance ratios. The intuition the paper offers is that a trigger pushes the prediction toward the attacker's target class, and added noise perturbs that push less than it perturbs an ordinary prediction. The paper supports this with a check on CIFAR-10: under σ = 1.0, the share of poisoned samples satisfying its noise assumption stays between 99.6% and 100% for Badnets, Blend, Trojan and Narcissus across ρ from 1 to 200.

Because the score is computed per input rather than from class-level or cluster-level structure, the authors argue it is less exposed to the distribution shifts that imbalance creates. That is a design claim supported by the detection results, not a separate experiment isolating the cause.

Conformal calibration turns scores into a bounded FPR

A score alone does not detect anything; a cutoff does. RPP computes empirical RPP scores on a small clean calibration set, then sets the threshold at the quantile tied to a significance level α. At detection time, any sample scoring at or below the threshold is labeled poisoned.

The calibration step uses split conformal prediction, which the paper describes as distribution-free and valid under class imbalance. This matters because the threshold is not tuned on the poisoned data being inspected, and the guarantee is attached to the false positive rate rather than asserted from the score distributions alone. The paper also states a certified condition: if the trigger's perturbation exceeds a threshold R derived from the noise level and the target-class probabilities, the poisoned sample is guaranteed detectable within the stated domain.

The guarantee carries assumptions the authors name: the calibration setting, the Gaussian perturbation model, and the considered data-poisoning threat model. They explicitly do not claim security against all adaptive or out-of-scope attacks. The calibration set must also be clean and reasonably representative; the paper notes that non-representative calibration samples could produce uneven false-positive behavior across classes, especially for underrepresented groups.

Detection holds up until imbalance becomes extreme

The headline comparison covers five benchmarks (MNIST, SVHN, CIFAR-10, TinyImageNet, ImageNet10), 10 backdoor attacks and 12 baseline defenses. The paper reports significantly higher detection accuracy than state-of-the-art defenses, particularly under imbalance. The most granular numbers come from the appendix ablations, which use true positive rate (TPR, the share of poisoned samples correctly flagged) and false positive rate (FPR, the share of clean samples wrongly flagged) as separate measures.

With as few as J = 3 noise samples, RPP exceeds 95% detection performance at ρ = 1, 2, 10 and 100. At ρ = 200 it keeps TPR above 85% with FPR below 40%. On SVHN at ρ = 200, where the score distributions visibly overlap, the method still reaches 90.0% TPR with FPR below 30%. The authors note that step imbalance at ρ = 200 is rare in real settings, which is a reason to read that row as a stress case rather than a typical deployment.

Architecture choice shifts the trade-off. Against Badnets on CIFAR-10 at ρ = 200, VGG16 reaches 90.0% TPR at 26.9% FPR, DenseNet161 reaches 88.9% at 30.6%, and EfficientNetB0 drops to 77.8% at 23.8%. On TinyImageNet, results are reported only up to ρ = 100, where VGG16 reaches 100.0% TPR at 36.6% FPR. Detection strength is therefore not uniform across backbones, and the FPR is not negligible in any of these rows.

Target class and rebalancing change the margin

Which class the attacker chooses matters. On imbalanced CIFAR-10 with a 0.3% poisoning ratio, RPP against Badnets reaches 99.4% TPR at 7.3% FPR at ρ = 2 when the target is minority class 9, and 93.9% TPR at 3.1% FPR when the target is intermediate class 4. At ρ = 200 the minority-target case holds at 90.0% TPR with 6.0% FPR, while the intermediate-target case falls to 79.6% TPR with 4.6% FPR. The lower-FPR rows are not automatically the better ones; they come with lower recall.

Rebalancing the training procedure also moves the numbers. Under logits adjustment, attack success rate at ρ = 200 drops from 99.7% to 94.3%, and RPP's TPR falls to 83.0% with 10.4% FPR, compared with 97.8% TPR at 26.1% FPR at ρ = 100. The authors read this as robustness to a weakened attack rather than a failure, since the detector still works when the backdoor itself is less effective.

Trigger size sets a practical floor. When the trigger is small enough that attack success rate falls below 50%, the paper states that defensive measures are not required. At a larger trigger norm of 3 on ρ = 100 and 200, RPP reaches 80% TPR with FPR under 30%; at norm 6 on ρ = 1, 2 and 10, TPR is 70.6%, 89.9% and 87.4% respectively with FPR below 15%.

A small clean calibration set is enough

The calibration set size n is the main practical cost, because it must be clean and drawn from a similar distribution to the inspected data. The paper reports that reducing n from 1000 to 100 leaves TPR nearly unchanged across datasets with varying imbalance, and that J = 3 noise draws suffice. Those two choices keep both the data requirement and the inference cost low relative to methods that need a large held-out clean set.

The trade-off to watch is the false positive rate, not just TPR. At α = 0.05 the reported FPRs range from roughly 3% to 40% depending on dataset, architecture, imbalance ratio and target class. A practitioner deciding whether to filter a training set should read the FPR for their own imbalance level and backbone, because the cost of discarding clean samples rises with that number.

The paper's own framing is that RPP is a step toward reliable defense under imbalance, not a complete solution to all poisoning threats. Its certified guarantees are scoped to the stated assumptions, and the authors flag dual-use risk: the stability-based detection principle could inform attackers designing more evasive triggers.

The imbalance ratio decides whether RPP is worth deploying

The paper's most decision-relevant split is by imbalance ratio. Up to about ρ = 100, RPP reports TPR above 95% with FPR under 40% using only three noise samples, and the calibration set can shrink to 100 clean examples without materially changing TPR. At ρ = 200, the score distributions overlap, TPR falls into the 78% to 90% range depending on backbone and target class, and FPR can approach 40%.

That makes the practical question less whether RPP beats the 12 baselines on average and more where a given deployment sits on that curve. If the dataset is moderately imbalanced and a small clean calibration set is available, the certified FPR bound plus sample-level scoring is the paper's distinctive offer. If imbalance is extreme or the trigger is small enough that attack success rate is already below 50%, the paper's own evidence suggests the detection margin narrows or the defense is not needed at all.

Sources used in this answer