Can multi-plane autoregressive diffusion synthesize 3D multi-contrast MRI?

MPAD reframes 3D synthesis as coordinated 2D slices

Multi-contrast MRI acquires several images of the same anatomy, each weighting tissue differently, and synthesizing a missing contrast from an acquired one would shorten scans. The paper treats this as an inherently 3D problem: the synthesized volume must stay anatomically coherent across axial, sagittal, and coronal planes. Fully 3D generative models can capture that structure, but the paper states their computational and memory requirements grow cubically with volume size, which becomes a practical barrier when one model must handle several target contrasts.

MPAD's answer is to keep the generation operations 2D while still producing a whole volume. A 3D autoencoder first compresses each MRI scan into an isotropic 3D latent representation, meaning the latent can be sliced consistently along any of the three anatomical orientations without geometric bias. A 2D diffusion model then works on individual latent slices rather than the full volume. The efficiency claim therefore comes from where the computation happens, not from shrinking the output.

Training reconstructs masked target slices from source context

During training, the source contrast and target contrast are both encoded into the latent space. For each iteration the model samples one plane orientation, slices the target latent along it, and masks a random subset of slice indices. The 2D diffusion model is trained to reconstruct those masked target slices while conditioned on the source-contrast slices, the unmasked target-contrast slices, and a text embedding naming the desired modality, such as a T2-weighted image. The text prompt is encoded with BioMedCLIP.

A 3D Multi-modal Conditioning Encoder fuses the source and partially masked target features, and the resulting conditioning is injected into each diffusion layer through spatially adaptive normalization (SPADE). Because the modulation parameters vary by spatial location, target-contrast cues can be applied locally rather than uniformly. Slices from different anatomical planes are batched together during training, so the denoising network is not tied to one orientation. This is the mechanism that lets a single unified model support one-to-many translation, meaning different target contrasts from the same source.

Inference generates slices in order, then reuses them as priors

At inference, MPAD factorizes the target volume into a sequence of 2D slices. Within one plane orientation, slices are generated autoregressively in a randomly sampled order, with each new slice conditioned on those already produced; the paper calls this Intra-plane Autoregressive Synthesis and credits it with maintaining continuity inside a plane. The first plane starts each slice from Gaussian noise at the maximum diffusion timestep.

The second mechanism, Inter-plane Prior Inference, carries completed volumes into the next orientation. For later planes, each slice is initialized not from pure noise but from a smaller intermediate timestep after forward-diffusing the corresponding slice of an already completed volume, then denoised from that point. After all three orientations are done, a refinement pass regenerates the first plane using slices from all three completed volumes. The four candidates are combined by voxel-wise averaging and decoded to the final image volume. Starting later planes from a smaller timestep is also how the method reduces sampling steps.

The efficiency gain is measured against 3D latent diffusion baselines

The headline comparison is computational. Relative to 3D latent diffusion baselines, MPAD reduces training FLOPs by 7x and inference FLOPs by 3x, and the paper also reports lower inference time and peak memory consumption. The abstract states that experiments on multiple datasets show superior performance, high-fidelity 3D volumes, and one-to-many translation within a single unified model. These are the paper's own reported results; the packet does not provide the per-dataset baseline table needed to check margins task by task.

The ablation tables give a more granular view of where the gains come from, averaged across six translation tasks (T1 to and from T2, T1 to and from PD, T2 to and from PD) on the ADNI and IXI datasets. Replacing the 2D conditioning encoder with a 3D convolutional one raised PSNR by 0.72 dB on ADNI and 1.19 dB on IXI. Replacing learnable mask tokens with Gaussian-noise masking raised PSNR by 0.32 dB and 2.00 dB respectively. Turning on inter-plane priors raised PSNR by 1.12 dB on ADNI and 1.88 dB on IXI. Because these are averaged over tasks and datasets, they describe aggregate behavior, not any single translation direction.

Adding planes helps more than choosing their order

The multi-plane ablation shows a consistent trend as views are added. A single coronal plane reached 22.82 dB PSNR on ADNI and 29.17 dB on IXI. Adding a second plane raised ADNI PSNR to 23.32 dB, and using all three planes reached 23.49 dB on ADNI and 29.29 dB on IXI. The final refinement pass on the first plane added a small further gain, to 23.54 dB on ADNI and 29.37 dB on IXI. The paper notes that two orthogonal planes give similar improvement regardless of which pair is chosen, which supports the interpretation that the benefit comes from complementary views rather than one privileged plane.

Plane ordering, by contrast, is nearly irrelevant in the reported test. Three different orderings on ADNI produced PSNR of 23.48, 23.49, and 23.54 dB, with SSIM of 0.830, 0.830, and 0.832. The authors explain this by noting that each plane is conditioned on both source information and previously synthesized volumes, so anatomical context accumulates regardless of sequence. For a reader weighing deployment choices, this suggests the ordering is not a sensitive knob, while the number of planes and the refinement pass are.

The autoregressive loop itself adds little in the ablation

One result complicates a simple story about autoregression. In the ablation on the autoregressive framework, removing it entirely gave 22.79 dB PSNR on ADNI and 28.89 dB on IXI. Adding it with 16 autoregressive groups gave 22.83 and 29.06 dB; with 4 groups, 22.85 and 29.08 dB; with 1 group, 22.87 and 29.14 dB. The direction is consistent but the magnitude is small, especially on ADNI, where the difference between no autoregression and the finest setting is 0.08 dB.

Read alongside the multi-plane table, this suggests the larger contributors in these ablations are the 3D conditioning encoder, Gaussian-noise masking, inter-plane priors, and multi-plane aggregation, rather than the sequential slice ordering inside a plane. That is an interpretation of the reported numbers, not a claim the authors make in those terms. It matters because a reader deciding what to reimplement should not assume the autoregressive factorization alone carries the quality gain.

The evidence supports a design trade-off, not a clinical claim

The evaluation is quantitative image fidelity on ADNI and IXI, reported through PSNR, SSIM, and NMSE averaged over six translation tasks. The packet does not include reader studies, diagnostic-task performance, or prospective clinical validation, so the results speak to synthesis fidelity and computational cost rather than to diagnostic substitution. The efficiency figures are also relative to the paper's chosen 3D latent diffusion baselines, so the 7x and 3x multipliers should be read as comparisons against those baselines, not as absolute costs.

The paper itself frames a controllable trade-off between inter-plane refinement and inference time, and lists distillation or parallelized multi-plane generation as future work to further reduce sampling cost. That framing is the practical takeaway: MPAD's value in this evidence is the ability to trade extra plane passes for coherence while keeping generation in 2D latent space. A reader comparing approaches should weigh the number of plane passes and the conditioning design, since those are the variables the ablations show moving PSNR, rather than the plane ordering, which the reported results show to be nearly invariant.

Sources used in this answer