Where does Mamba actually beat Transformers?
Mamba's biggest advantage is its ability to handle extremely long sequences efficiently. Unlike Transformers, whose attention mechanism scales quadratically with sequence length (doubling the input quadruples the compute), Mamba scales linearly — so doubling the input only doubles the compute. This lets Mamba process documents several times longer than Transformers can, while maintaining or even surpassing performance on tasks like legal document classification and case law retrieval [2].
On standard language modeling, Mamba is also remarkably competitive. The original Mamba-3B model outperforms Transformers of the same size and matches Transformers twice its size (e.g., a 6B-parameter Transformer) in both pretraining and downstream evaluation [4]. It also achieves 5x higher inference throughput than Transformers, meaning it can generate text five times faster on the same hardware [4].
So what's the catch — why isn't everyone switching to Mamba?
Mamba isn't universally better. On text reranking — a task that requires fine-grained understanding of how a query relates to a document — Mamba models match Transformer performance, but they are actually less efficient in both training and inference compared to Transformers that use flash attention (a hardware-optimized version of attention) [3]. This means for shorter sequences or tasks where attention is already fast, Mamba offers no speed benefit and may be slower.
Another limitation is that Mamba's design, which makes it so good at long sequences, also makes it worse at certain kinds of content-based reasoning that Transformers handle naturally. The original Mamba paper explicitly identifies this as a key weakness of earlier state space models and introduces 'selective' mechanisms to address it [4]. Even with those improvements, the best performance on many language tasks still comes from hybrid models that combine Transformer encoders with Mamba decoders, suggesting each architecture has complementary strengths [1].
Is the real breakthrough a hybrid of both?
The evidence points toward hybrid models as the most practical path forward. A 2025 study found that combining a Transformer encoder with a Mamba decoder, plus a feature fusion technique that blends outputs from both, consistently outperformed existing benchmarks across various language tasks [1]. This suggests that rather than one architecture replacing the other, the best NLP systems will likely use both — Transformers for their powerful encoding and content-based reasoning, and Mamba for efficient decoding and long-context handling.
Even within the Mamba family, newer versions improve on older ones. Mamba-2 outperforms Mamba-1 in both performance and efficiency on text reranking tasks [3], showing the architecture is still evolving rapidly. The bottom line: Mamba is a genuine breakthrough for long-context and efficiency-critical applications, but it's not a silver bullet — and the biggest gains may come from smart combinations of both approaches.
Sources used in this answer
A hybrid model based on transformer and Mamba for enhanced sequence modeling
A hybrid Transformer-Mamba model (Transformer encoder + Mamba decoder with feature fusion) consistently outperformed existing benchmarks across various language tasks.
Scaling Legal AI: Benchmarking Mamba and Transformers for Statutory Classification and Case Law Retrieval
Mamba processes legal documents several times longer than Transformers while maintaining or surpassing classification and retrieval performance, thanks to its linear-time scaling.
State Space Models are Strong Text Rerankers
Mamba architectures match Transformer performance on text reranking but are less efficient in training and inference than Transformers with flash attention; Mamba-2 outperforms Mamba-1.
Mamba: Linear-Time Sequence Modeling with Selective State Spaces
Mamba-3B outperforms Transformers of the same size and matches Transformers twice its size in language modeling, with 5x higher inference throughput and linear scaling to million-length sequences.
First-order State Space Model for Lightweight Image Super-resolution
A modified Mamba module (FSSM) improved image super-resolution performance on five benchmark datasets without increasing parameters, showing SSMs can be enhanced beyond their original design.