[TPAMI 2025] BaryIR: Mastering Universal Image Restoration via Wasserstein Barycenter Space

TL;DR

BaryIR is a high-performance "All-in-One" image restoration framework that breaks the limitation of earlier models which struggled with unseen (OOD) degradations. By mapping multisource degraded features into a Wasserstein Barycenter (WB) space, it captures the "invariant geometry" shared across all corruptions, allowing the model to generalize to entirely new types of noise, blur, or weather conditions.

The Core Motivation: Beyond Simple Parameter Sharing

Most existing All-in-One Image Restoration (AIR) models treat different degradations (noise, rain, haze) as distinct tasks to be "conditioned" into a single network. While effective on training data, they fail when they encounter a degradation type or level they haven't seen before.

The Insight: The authors argue that while degradations vary, the underlying "clean" content remains the same. If we view different degradations as specific "shifts" from an underlying distribution, finding the geometric center of these shifted distributions should yield a representation that is robust to any specific corruption.

Methodology: The Geometry of Neutrality

BaryIR introduces two key architectural innovations to achieve this:

1. The Continuous Wasserstein Barycenter Map

Instead of discrete prototypes, BaryIR uses a neural network to learn a continuous map $T_{h}eta$ that transports features from the degraded multisource space into the WB space. This space is defined by the distribution $Q$ that minimizes the weighted sum of Wasserstein distances: ${\inf }_{Q\in \mathcal{P}({\mathcal{Z}}_B)}{\sum }_{k=1}^K{\lambda }_{k}W({\mathbb{P}}_k,Q)$ By solving this via a max-min adversarial optimization, the model effectively filters out "degradation-specific" noise and focuses on the shared structural commonalities of natural images.

2. Disentangled Residual Subspaces

To ensure that valuable degradation-specific cues aren't lost (as they help in targeted restoration), the authors define Residual Subspaces.

• Inter-residual contrastive (IRC) loss: Ensures different degradation residuals are distinct.
• Barycenter-residual orthogonal (BRO) loss: Forces the residual features to be orthogonal to the WB features. This creates a clean "separation of concerns" within the hidden layers of the network.

Experimental Excellence

The performance of BaryIR is particularly striking when tested on data it was never trained on:

• Unseen Degradation Levels: When trained on noise levels up to $\sigma =50$ and tested on $\sigma =75$, BaryIR outperformed the next best method by 2.20 dB.
• Unseen Real-World Data: On the O-HAZE dataset (real outdoor haze), BaryIR achieved a PSNR of 22.98 dB, a 2.09 dB lead over MoCE-IR.
• Mixed Degradations: It handles complex scenarios (e.g., simultaneous rain and haze) significantly better by relying on the learned "clean commonalities."

Efficiency Analysis

Despite the mathematical complexity, BaryIR is highly efficient. It adds only ~8.3M parameters and a mere 0.03s of inference latency over the standard Restormer backbone, making it vastly more practical than vision-language-based models like DA-CLIP.

Critical Insight: The Value of Invariance

The success of BaryIR lies in its Inductive Bias. By forcing representations to live in a Barycenter space, the model is architecturally prevented from over-specializing in one single type of corruption. This "forced neutrality" is what grants it its formidable generalization powers.

Conclusion & Future Look

BaryIR marks a shift from "conditioning" models toward "alignment-based" models in image restoration. The authors suggest that this Wasserstein-driven approach could extend to Multimodal Learning, where the Barycenter could represent a truly modality-agnostic "concept" space shared between text, audio, and vision.

Limitations: In cases of extreme localized intensity (e.g., heavy rain streaks indistinguishable from structure), the model can still fail. Improving focal sensitivity in the residual space remains an open challenge.