[Interspeech 2026] MUGEN: Unveiling the Scaling Bottleneck in Multi-Audio Understanding

TL;DR

While Large Audio-Language Models (LALMs) have mastered single-clip tasks, MUGEN reveals a startling weakness: their performance collapses when asked to compare multiple audio files simultaneously. By introducing a benchmark of 35 tasks across 7 dimensions, researchers found that current models suffer from severe positional bias and an "input scaling" problem. The solution? Audio-Permutational Self-Consistency (APSC)—a training-free strategy that boosts accuracy by up to 6.74%.

The Blind Spot of Modern LALMs

In the current AI landscape, LALMs like Qwen2.5-Omni and Gemini-3-pro are often touted for their "human-like" hearing. However, most benchmarks only test them on one clip at a time (e.g., "What is the person saying?").

In the real world, we need models to handle:

• Speech RAG: Finding a specific fact across multiple recorded meetings.
• Audio In-Context Learning: Learning a new task from 10 different audio examples.
• Comparison Tasks: "Which of these five speakers sounds most like the reference?"

MUGEN proves that even the best models today struggle once the "audio-count" increases, identifying a critical gap between isolated perception and integrated reasoning.

Methodology: The MUGEN Framework

MUGEN shifts the paradigm by using an audio-as-option design. Instead of selecting a text answer, the model is given a text instruction and must select the correct audio file from a list of candidates.

1. Seven Dimensions of Hearing

The benchmark covers everything from Semantics (what is said) to Acoustic Scene Analysis (environmental sounds) and Affective State (emotion). Crucially, it emphasizes non-semantic traits to ensure models aren't just using "textual shortcuts" via internal ASR.

2. The Scaling Challenge

The researchers systematically tested models with 2, 3, 4, and 5 audio inputs. The results were clear: as complexity grows linear, the performance drops exponentially for many open-source models.

Figure 1: Task distribution across the seven evaluation dimensions in MUGEN.

Why Does Performance Degrade?

The study points to two primary culprits:

1. Acoustic Bottleneck: Models fail to differentiate subtle acoustic features (like emotion or pitch) across different files.
2. Positional Bias: Models often favor audio files placed at the beginning or end of a prompt, rather than reasoning through all options equally.

Interestingly, Chain-of-Thought (CoT)—the holy grail of LLM reasoning—failed to help much here. Why? Because the problem isn't logic; it's perception. If the model "hears" the wrong emotion, no amount of step-by-step thinking will lead to the right answer.

Improving the "Ear" without Retraining: APSC

To fix this without an expensive training run, the authors introduced Audio-Permutational Self-Consistency (APSC).

The logic is elegant:

1. Permute the order of the audio options 10 different times.
2. Ask the model the same question for each permutation.
3. Aggregate the answers using a majority vote.

By shuffling the deck, the model is forced to overcome its positional preference, resulting in significant jumps in accuracy.

Table 1: Performance gains using APSC and CoT. Notice how APSC consistently outperforms standard Self-Consistency (SC).

Critical Insights & Takeaways

• Proprietary vs. Open Source: Gemini-3-pro significantly outperforms open-source models, but even it struggles with "Temporal Awareness" (judging audio duration and pacing).
• The Scalability Wall: LALMs are currently sensitive to the number of tokens/embeddings generated by multiple audio clips. We need more efficient "Audio Summarizers" within the architecture.
• Future Work: The industry must move beyond "Speech-to-Text" thinking. Multi-audio understanding requires the model to treat audio as a first-class citizen, preserving paralinguistic cues that cannot be captured in a transcript.

The Bottom Line: MUGEN provides the first rigorous yardstick for complex auditory reasoning, showing that the path to true "Audio General Intelligence" requires models that can not only hear but also compare and contrast.