DVD: Rethinking Sparse Voxel Generation with Discrete Diffusion

TL;DR

Sparse voxel scaffolds are the backbone of hybrid 3D generation (SLAT). Traditionally, these are generated via continuous diffusion and then "forced" into binary values. DVD (Discrete Voxel Diffusion) flips this script by modeling voxels as native discrete variables. The result? No more "holes" in thin structures, an explicit uncertainty metric to find "difficult" samples, and a powerful new way to edit 3D assets via localized perturbations.

The "Binarization" Pain Point

The current state-of-the-art in 3D generation often follows a two-stage process: first, generate an explicit sparse scaffold (voxels), then attach high-dimensional latents to these voxels to decode the fine surface.

The problem lies in Stage 1. Models like TRELLIS generate continuous values that are thresholded at zero. This creates a geometric "uncanny valley" where thin layers (like the sides of a container or a chair leg) fail to resolve correctly. If the continuous field is slightly off near the zero boundary, the voxel disappears—creating the infamous "hole" artifact.

Methodology: Embracing Discreteness

DVD avoids the continuous-to-discrete conversion entirely. It utilizes Uniform State Discrete Diffusion Models (USDM). Instead of predicting a Gaussian mean, the model outputs a categorical distribution $q(x_0|x_t)$ for every grid position.

1. Model Architecture

The core is a DiT-based (Diffusion Transformer) architecture that processes the 3D grid as a sequence. DVD is trained to denoise voxels by predicting their final occupied/empty state directly.

Figure 1: Comparison of the standard SLAT pipeline vs. the DVD approach (Bottom), showcasing the categorical transition of voxels from noise to clean states.

2. Predictive Uncertainty

Because DVD outputs a probability distribution, we can calculate the Predictive Entropy.

• Physical Intuition: If the model is 50/50 on whether a voxel belongs to a surface, the entropy is high. This usually happens at sharp edges or thin structures.
• Application: The authors aggregate this into a shape-level score ${\gamma }_{ϵ,\rho }$ to identify complex models and filter training data.

Experiments & Results

DVD was tested against industry-standard baselines like TRELLIS and TripoSR.

Quantitative Edge

On the Toys4K dataset, DVD showed a significant reduction in Chamfer Distance (CDV) compared to the continuous version (0.0085 vs 0.0150). This indicates that the generated geometry is much closer to the ground truth.

Figure 2: Qualitative comparison showing how continuous methods (left) produce disconnected holes in thin structures, whereas DVD (right) maintains structural integrity.

Efficient Voxel Editing (BSP)

The paper introduces Block-Structured Perturbations (BSP) for fine-tuning. Instead of uniform noise, the model is trained to "fill in the blanks" of axis-aligned cubes. This allows for high-quality inpainting—for example, changing a house's roof from a "wizard hat" to a "chimney" style—within a single sampling round.

Critical Insight & Conclusion

The success of DVD signals a shift in 3D research. While continuous diffusion (Gaussian) dominates image and video generation, the inductive bias of discrete modeling is inherently better suited for structural spatial representations like voxels.

Takeaway: If your data is binary or categorical by nature, don't force it into a continuous latent space. Directly modeling the probability of state transitions not only improves quality but gives you an "uncertainty map" that is invaluable for professional 3D editing workflows.

Limitations: The sampling speed (NFE=256) is still a hurdle compared to single-step or few-step GAN-based methods. Future work in distillation will be key to making DVD real-time.