[ICML 2024] Generalized Bayes for Causal Inference: A New Frontier for Robust Uncertainty Quantification

TL;DR

Quantifying uncertainty in causal effects (like "Does this drug work for everyone?") usually requires building a full, fragile model of how the data was born. This paper flips the script: it proposes a Generalized Bayesian framework that puts priors directly on the causal effect itself. By using Neyman-orthogonal losses, the authors create a "bulletproof" posterior that remains accurate even when the underlying machine learning models for propensity scores are slightly off.

Problem & Motivation: The "Likelihood" Trap

In traditional Bayesian causal inference, if you want to estimate the Average Treatment Effect (ATE), you are forced to model the entire joint distribution of covariates, treatments, and outcomes. This is a nightmare in high dimensions.

• The Problem: You have to place priors on "nuisance" functions (complexity you don't care about, like the probability of receiving a treatment).
• The Consequence: These priors can interact in weird ways, causing regularization-induced confounding. If your nuisance model is wrong, your whole posterior distribution for the causal effect becomes untrustworthy.

The authors' insight is rooted in Decision Theory: we don't need a likelihood if we have an identification-driven loss function. By updating our beliefs through a loss (Gibbs posterior), we focus only on the causal parameter of interest.

Methodology: The Power of Orthogonality

The core of the method is the Gibbs Posterior update: $q_n(heta|{\mathcal{D}}_n)\propto \exp \{-\omega n{\mathcal{L}}_n(heta;\hat{\eta })\}\pi (heta)$

Where ${\mathcal{L}}_n$ is the loss, $\pi (heta)$ is the prior on the effect, and $\omega$ is a calibration parameter.

Why Neyman-Orthogonality Matters

Simply using any loss isn't enough. If you use a "naive" loss, errors in your nuisance estimates ($\hat{\eta }$) leak into your posterior. The authors advocate for Neyman-orthogonal losses (like AIPW). These losses are "flat" with respect to nuisance perturbations, meaning the gradient of the loss doesn't change much if the propensity score is slightly wrong.

Figure 1: The workflow integrates prior elicitation with cross-fitted ML losses to produce calibrated posteriors.

Experiments & Results

The authors tested their framework against 9 synthetic datasets ranging from simple linear models to complex "Friedman-style" nonlinearities.

Key Findings:

1. AIPW is King: The Neyman-orthogonal AIPW strategy consistently hit the ~95% coverage target for credible intervals, whereas non-orthogonal methods (RA, IPW) often failed or produced "unfaithful" intervals.
2. Efficiency: Not only were the intervals accurate, they were the narrowest possible, meaning the model is both "certain" and "correct."

Figure 2: As sample size increases, the AIPW credible intervals (red) shrink faster and more reliably than baseline methods.

For Conditional Average Treatment Effect (CATE), the framework supports Gaussian Processes (GPs). Using the DR (Doubly Robust) loss, the GP can map out treatment effects across different patient characteristics with full, calibrated uncertainty bands.

Figure 3: Visualization of the CATE function fit. The green line represents the ground truth, perfectly captured by the model's uncertainty bands.

Critical Analysis & Conclusion

Takeaway

This work democratizes Bayesian Causal Inference. It allows researchers to keep their favorite ML "meta-learners" (like DR-Learners) and simply wrap them in this generalized framework to get rigorous uncertainty quantification.

Limitations

• Calibration Dependence: The method relies on a calibration parameter $\omega$. While the authors use a bootstrap tuning method, this adds computational overhead.
• Nuisance Consistency: While robust to rates of convergence, the method still requires the nuisance estimators to be consistent. It cannot fix a fundamentally biased nuisance model that never converges.

Future Outlook

The marriage of Orthogonal Statistical Learning and Generalized Bayes opens the door for robust causal discovery in higher-risk fields like medicine and public policy, where knowing "I don't know" is as important as the estimate itself.