[CVPR 2024] FREE-Edit: Precision Video Editing via Editing-aware Feature Injection

TL;DR

FREE-Edit is a training-free, zero-shot framework for image-driven video editing. By leveraging the efficiency of Rectified Flow (RF) models and introducing an Editing-awaRE (REE) injection mechanism, it solves the "content leakage" problem common in video propagation. It tracks edited regions via optical flow and adaptively masks feature injection, ensuring that edits stay crisp while the background motion remains perfectly undisturbed.

Background: The Injection Dilemma

In image-driven video editing, the goal is to take a source video, modify the first frame (e.g., using Photoshop), and have that modification realistically "flow" through the rest of the video.

The standard approach uses an "inversion-then-editing" pipeline. During the denoising (editing) phase, internal representations (Query and Key) from the original video's reconstruction are "injected" into the process to maintain the original motion. However, this creates a conflict:

• Excessive Injection: The original pixels "leak" back into the edited area, ruining the modification.
• Insufficient Injection: The video loses its original motion, leading to flickering or drifting.

Methodology: Decoupling Motion from Modification

FREE-Edit introduces a surgical approach to feature injection. Instead of applying the same injection weight to every pixel, it uses a modulation weight ($\lambda$).

1. The REE Injection Mechanism

The core identity of a Rectified Flow transformer block is its self-attention. FREE-Edit modifies the Query ($\bar{Q}$) and Key ($\bar{K}$) as follows: $\bar{Q}=\lambda Q+(1-\lambda )ildeQ$ $\bar{K}=\lambda K+(1-\lambda )ildeK$ Where $Q,K$ are from the original video (reconstruction) and $ildeQ,ildeK$ are from the edited path.

2. Optical Flow Masking

To calculate $\lambda$, FREE-Edit:

1. Detects Edits: Compares the source first frame and edited first frame to create a binary mask.
2. Tracks Motion: Uses RAFT to estimate optical flow and warps that mask across all frames.
3. Inverts for Injection: Sets $\lambda \approx 1$ for non-edited areas (preserve motion) and $\lambda =0$ for edited areas (allow new content).

Figure 1: The FREE-Edit pipeline showing the "inversion-then-editing" flow and the REE injection module.

Experiments & SOTA Comparison

The authors tested FREE-Edit on I2V-Edit-Bench, a new benchmark covering object replacement, style transfer, and background editing.

Visual Superiority

Compared to previous methods like AnyV2V or VideoShop, FREE-Edit maintains significantly higher texture detail and temporal stability. While "Vanilla Injection" often results in the original object's ghosting appearing over the edit, REE injection keeps the edited area clean.

Figure 2: Qualitative comparison showing how REE injection avoids content leakage vs. vanilla methods.

Performance Metrics

• Non-editing Area Fidelity: Achieved a PSNR of 27.05, proving the background remains nearly identical to the source.
• Speed: Processing a 41-frame video in just 1.5 minutes on an A100—nearly 80x faster than training-based methods like I2VEdit.

Critical Analysis & Conclusion

FREE-Edit effectively bridges the gap between high-quality Rectified Flow models and practical video editing requirements.

Takeaway: The "editing-aware" philosophy is a powerful Inductive Bias. By acknowledging that different regions of a video require different levels of "guidance" from the source, we can bypass the need for expensive fine-tuning.

Limitations: Since the method relies on propagating the first frame's appearance along the source video's motion paths, it struggles with newly added objects that require unique trajectories (e.g., adding a turtle that should swim differently than the water flows). Future work integrating trajectory control would likely resolve this.

Future Outlook

As Rectified Flow models like LTX-Video and Wan2.1 become the industry standard, plug-and-play modules like FREE-Edit will be essential for making professional-grade video editing accessible to non-technical users.