Why do long documents slow down traditional OCR?
The core problem is that most modern OCR models generate text one token at a time, and they keep a running 'memory' (called a KV cache) of everything they've generated so far. As the document gets longer, this cache grows, eating up memory and making each new token slower to produce. This is a stark contrast to humans, who don't slow down when copying a long text [1]. This is why a standard OCR model might handle a page fine but choke on a 50-page book.
One paper highlights this exact issue: as the output sequence lengthens, the accumulated KV cache drives up memory consumption and progressively slows down generation [1]. This is the fundamental challenge that any solution must address.
How are researchers solving the long-document problem?
Two main strategies are emerging: making the memory usage constant regardless of length, and generating multiple tokens at once instead of one by one. The first approach, demonstrated in a model called Unlimited OCR, replaces the standard attention mechanism with a 'Reference Sliding Window Attention' that keeps the KV cache at a constant size throughout decoding [1]. This allows the model to transcribe dozens of pages in a single forward pass under a standard 32K token limit—a huge leap in practical capacity.
The second strategy uses diffusion models to generate text in parallel blocks. A model called DODO does this, achieving up to 5x faster inference than autoregressive baselines, while maintaining near state-of-the-art accuracy [3]. The key insight is that OCR is a deterministic task—the visual input dictates the output—so you don't need to generate sequentially. However, this approach requires careful design to avoid errors in the rigid, exact-match requirements of OCR [3].
Does scaling work for all types of documents?
The answer depends on the document's condition and language. For clean, modern documents, the new architectures work well. But for degraded historical documents, accuracy drops, and you need additional preprocessing. A pipeline called PreP-OCR combines image restoration with post-OCR correction, reducing character error rates by 63.9-70.3% compared to OCR on raw images across 13,831 pages of real historical documents in English, French, and Spanish [2]. This shows that scaling to long documents isn't just about architecture—it's also about handling the quality of the input.
Similarly, for languages with very long words, like Sanskrit, standard OCR struggles. A specialized model using attention-based LSTM achieved a word error rate of 15.97% and a character error rate of 3.71% on challenging Indic texts [4]. This suggests that while the length problem is being solved, language-specific challenges remain, and a one-size-fits-all approach may not work for every corpus.
About These Sources
This answer is built on 4 studies (1 peer-reviewed, 3 preprints) — published from 2020 to 2026, 3 from 2024 or later — selected as the most relevant from 4 studies that passed quality screening, drawn from 66 papers retrieved from a database of over 500 million.
Sources used in this answer
Unlimited OCR Works
Unlimited OCR introduces Reference Sliding Window Attention to keep KV cache constant, enabling transcription of dozens of pages in a single forward pass under a 32K token limit, and is also applicable to ASR and translation.
PreP-OCR: A Complete Pipeline for Document Image Restoration and Enhanced OCR Accuracy
PreP-OCR, a two-stage pipeline with image restoration and post-OCR correction, reduces character error rates by 63.9-70.3% on 13,831 pages of real historical documents in English, French, and Spanish.
DODO: Discrete OCR Diffusion Models
DODO, a discrete diffusion OCR model, achieves up to 5x faster inference than autoregressive baselines while maintaining near state-of-the-art accuracy, by generating text in blocks to avoid synchronization errors.
An OCR for Classical Indic Documents Containing Arbitrarily Long Words
A Sanskrit-specific OCR using attention-based LSTM achieves a word error rate of 15.97% and a character error rate of 3.71% on challenging Indic texts, addressing the issue of arbitrarily long words.