[CVPR 2025] StreamReady: Bridging the Gap Between What to Answer and When in Long-Video Streams

TL;DR

Streaming video understanding is often a race against time. While current models focus on correctness, they often fail at timing—either guessing before the evidence appears or trailing far behind it. StreamReady introduces the Answer Readiness Score (ARS) to penalize speculative early guesses and delayed responses. By using a hierarchical memory tree and a specialized readiness token, it learns to "watch and wait," only triggering the LLM once sufficient evidence is retrieved.

Background: The Timing Crisis in Streaming AI

In real-world streaming (e.g., watching a security feed or a live sports match), questions are often proactive. For example: "How many times will the child jump?" asked before the jumps even start.

Current SOTA models (offline or semi-online) fail here for two reasons:

1. Speculation (Early Penalty): They try to answer immediately based on priors, leading to hallucinations.
2. Delay (Late Penalty): They lack an internal "trigger" and only process frames as a passive buffer, missing the critical window for real-time utility.

Methodology: The "Watch and Wait" Architecture

StreamReady operates on three core pillars: Memory, Reasoning, and the Readiness Gate.

1. Hierarchical Visual Memory Tree

To handle 60-minute streams without OOM (Out of Memory) errors, the authors abandoned full frame-buffer approaches for a three-layer abstraction:

• L1 (Raw Frames): A FIFO buffer for immediate visual persistence.
• L2 (Centroids): Clustered embeddings representing mid-term events.
• L3 (Prototypes): High-level coarse summaries for long-range context.

2. Query-Aware Reasoning

The model doesn't just look at frames; it looks for evidence. A dual-branch Q-Former performs coarse-to-fine retrieval, using the question to "lookup" relevant prototypes and centroids.

3. The Readiness Mechanism (<RDY> Token)

This is the "secret sauce." Instead of a complex auxiliary model, they append a learnable <RDY> token to the reasoning representation. A lightweight head monitors this token's alignment with the query. If the evidence is sharp and consistent, the readiness score peaks, and the LLM is gated "Open."

ProReady-QA: A New Standard for Timing

The authors identified that existing benchmarks (like VStream-QA) only test "past-dependent" questions. They introduced ProReady-QA, featuring:

• 5,000 QA Pairs: Strictly proactive (future-dependent).
• Annotated Evidence Windows: Precise timestamps [Start, End] for when an answer becomes knowable.
• ARS Metric: A timing-aware score with asymmetric penalties—punishing early guesses (sigmoid-based) more harshly than slight delays.

Experimental Performance: Accuracy + Readiness

StreamReady achieves a dominant position across multiple benchmarks, particularly excelling in the "Effective Accuracy" (Acc × ARS).

Key Findings:

• Superior Timing: In tasks like Repetitive Event Count (REC) and Goal-State Detection (GSD), StreamReady saw ARS gains of nearly 20% over baselines.
• Efficiency: Unlike standard MLLMs that exhibit quadratic latency growth as the video progresses, StreamReady’s fixed-size hierarchical memory ensures constant inference time, essential for hour-long streams.

Critical Insight & Conclusion

StreamReady proves that "Readiness" is not just a heuristic—it can be a learned latent property. By stopping gradients from the readiness signal back to the main reasoning module, the authors ensure that the model doesn't "cheat" on answer correctness to satisfy timing constraints.

Takeaway: Future AI assistants in robots and smart homes will need to master the art of silence until the moment they are sure. StreamReady provides the first rigorous mathematical and architectural framework to achieve this "on-time" intelligence.