Eulerian Motion Guidance: Overcoming Texture Drift in Long-Horizon Image Animation

TL;DR

Current image animation models often suffer from "identity melting" or texture sliding because they try to track motion relative to a single starting frame. This paper introduces Eulerian Motion Guidance (EMG), which shifts the perspective to local, frame-to-frame transitions. Combined with a clever Bidirectional Geometric Consistency (BGC) check to handle occlusions, this method produces videos that remain sharp and structurally sound even over long sequences (100+ frames), while training 2.7x faster.

The Problem: The Lagrangian Trap

Most controllable Video Generation (I2V) frameworks use a Lagrangian approach. Imagine trying to describe the movement of every person in a marathon by measuring their distance from the starting line. As the race progresses, your measurements become more prone to error, and once runners go around a corner (occlusion), your tracking fails.

In video diffusion, this manifests as:

1. Error Accumulation: Small errors in optical flow estimation grow exponentially as the video progresses.
2. Supervisory Sparsity: When an object rotates or moves behind another, the "reference" disappears. The model is left guessing, which creates "ghosting" or "hallucinations."

The authors mathematically prove that Lagrangian error is lower-bounded by $\sigma \sqrt{t}$, meaning it must diverge over time.

Methodology: The Eulerian Alternative

Instead of looking back at frame 0, Eulerian Motion Guidance focuses on the flux—how pixels move between frame $t$ and $t+1$.

1. Bounded Error Flux

By focusing on short temporal hops, the motion signal stays within the high-fidelity linear regime of flow estimators (like RAFT). Theorem 2 in the paper proves this keeps the supervisory error uniformly bounded, preventing the "drift" found in earlier models.

2. Bidirectional Geometric Consistency (BGC)

Local tracking has one weakness: stochastic drift. Without an anchor, the background might start "sliding" with the foreground. The authors solve this with a Cycle Energy Formulation.

• They compute flow from $tot+1$ and then backwards from $t+1ot$.
• If a pixel doesn't end up where it started, it’s likely an occlusion or an error.
• They generate a dynamic Occlusion Mask to "mute" the training signal in these areas, ensuring the model only learns from geometrically valid data.

Figure: The EMG Pipeline featuring the Sparse-to-Dense module and the BGC masking process.

Experiments & SOTA Performance

The model was tested on trajectory-based (dragging objects) and keypoint-based (facial animation) tasks.

• Long-Horizon Stability: While baselines like MOFA show the subject "blurring out" by frame 100, EMG keeps textures like fur and skin sharp.
• Warping Error: EMG achieved a 28% reduction in warping error compared to ImageConductor, proving its pixel-level stability.
• Efficiency: By treating the temporal dimension as a batch, they parallelized flow computation, achieving a near O(1) complexity relative to sequence length.

Figure: Comparison between Lagrangian (MOFA) and Eulerian (Ours). Note the preservation of the bear's texture at frame 100.

Critical Analysis & Conclusion

Takeaway: This work represents a shift toward more physically grounded motion control. By acknowledging the mathematical limitations of reference-anchored tracking, the authors provide a robust blueprint for high-fidelity, long-form video generation.

Limitations:

• The reliance on a pre-trained flow estimator (RAFT) means the model is only as good as its underlying optical flow logic.
• Extreme motions might still challenge the BGC thresholding, potentially leading to localized blurring if $a_1$ and $a_2$ are not perfectly tuned.

Future Outlook: Integrating this Eulerian perspective into Video Diffusion Transformers (DiTs) could be the next jump, further scaling the complexity of scenes that can be animated from a single image.