TL;DR

While modern Text-to-Image (T2I) models like FLUX and SD3.5 produce stunning visuals, they often fail at "spatial logic"—placing objects in the wrong spots or ignoring orientation. SpatialReward introduces a verifiable reward pipeline that decomposes prompts, uses expert "safety" detectors, and applies Chain-of-Thought (CoT) reasoning to provide a precise training signal. The result? A massive leap in spatial consistency without losing aesthetic quality.

The Problem: Appearance is High, Logic is Low

Current Reinforcement Learning (RL) for diffusion models typically uses holistic scorers like CLIP or HPSv2. While these make images "look good," they are notoriously bad at counting, positioning, and attribute binding. If you ask for "a book on the left and a pen on the right," a standard reward model might give a high score just because both objects are present, even if their positions are swapped. This "spatial blindness" limits the utility of T2I in professional design and complex storytelling.

Methodology: Perception Meets Reasoning

The core philosophy of SpatialReward is that reward models should be as verifiable as a math problem. The authors break the black-box evaluation into three interpretable steps:

1. Prompt Decomposition

Instead of passing the whole prompt to a VLM, the system uses a fine-tuned Qwen2.5-VL-7B to extract a structured "Constraint Set." It identifies exactly what objects should be there, their counts, and their relative spatial attributes (e.g., Sinks: 4, Inscribed: "Clean").

2. Expert Grounding (The "Verified" Signal)

To avoid VLM hallucinations, the model calls in "Expert Detectors."

• YOLO-World/DINO for bounding boxes.
• Depth Anything for 3D order.
• PaddleOCR for text accuracy and location.

This turns visual data into factual metadata (Coordinates, Depth-ranks, Color-sims).

3. Spatial CoT Reasoning

Finally, the model feeds the bounding boxes and detection scores back into a VLM. Instead of asking "Is this image right?", it asks "Based on these detected boxes [x,y] for Object A and [x,y] for Object B, does the 'on-top-of' relationship hold?" This reasoning step filters out noise and understands nuances like "behind" vs "under."

Figure 1: The SpatialReward pipeline—from free-form prompt to verifiable CoT reward.

Quantitative Breakthroughs: SpatRelBench

The authors also introduced SpatRelBench, a benchmark testing 1,000+ objects across orientations, 3D positioning, and text placement.

The results speak for themselves:

• Positioning Accuracy: Jumped from 0.28 (SD3.5 baseline) to 0.98 using SpatialReward.
• Counting Consistency: Significant gains across both SD and FLUX backbones.
• Human Alignment: SpatialReward achieved a Spearman correlation of 0.63, significantly higher than CLIPScore (0.42) or ImageReward (0.48).

Table 1: Performance metrics across GenEval and SpatRelBench benchmarks.

Visual Evidence

As seen in the qualitative samples, SpatialReward effectively forces the model to respect text inscriptions (e.g., correctly labeling sinks) and complex positioning (e.g., a man standing beside a vending machine rather than merging into it).

Figure 2: SpatialReward vs. Baselines. Note the superior text placement and object orientation.

Critical Insight: Why Does This Work?

The Ablation Study reveals that removing Expert Detection causes the biggest performance drop. This confirms a vital trend in AI: LLMs/VLMs are great reasoners but mediocre observers. By providing the "reasoner" with high-quality "observations" from specialized detectors, we solve the hallucination gap. Furthermore, the Exclusion Constraints (negative rewards for unwanted objects) prevent the model from "cheating" by just throwing every mentioned object randomly onto the canvas.

Conclusion

SpatialReward shifts the paradigm from "evaluating by looking" to "evaluating by verifying." For developers and researchers using RL to fine-tune diffusion models, this paper proves that the bottleneck isn't the RL algorithm (like GRPO), but the granularity and truthfulness of the reward signal.

Limitations: There is a slight trade-off in "Aesthetic" scores, suggesting that focusing too heavily on "logical correctness" might slightly dampen the artistic flair of models like SD3.5, a common challenge in RLHF/RLAIF.