[CVPR 2025] Mario: Beyond Isolated Pairs with Modality-Adaptive Graph Reasoning
Mario: Multimodal Graph Reasoning with Large Language Models
The paper introduces Mario, a unified framework for multimodal graph (MMG) reasoning using Large Language Models (LLMs). It achieves state-of-the-art performance across node classification and link prediction tasks by combining structure-aware cross-modal alignment with a modality-adaptive instruction tuning mechanism.
TL;DR
Mario is a novel framework that brings the power of Large Language Models (LLMs) to Multimodal Graphs (MMGs). Unlike prior works that treat image-text pairs as independent units, Mario leverages graph topology to fix cross-modal inconsistencies and introduces a "Modality-Adaptive Prompt Router" to choose whether the LLM should look at text, images, or both for any given node. The result? A massive leap in SOTA and zero-shot generalization.
The Problem: The "Isolated Pair" Fallacy
Most Multimodal LLMs (like LLaVA) are trained on isolated image-caption pairs. However, real-world data is rarely that clean. Consider an E-commerce graph: a product image might show a logo while the text describes the material.
The authors identify two fatal flaws in existing methods:
- Weak Cross-modal Consistency (C1): The text is often not a faithful caption of the image.
- Heterogeneous Modality Preference (C2): Some nodes (like music CDs) are text-salient, while others (like apparel) are vision-salient. A fixed prompting strategy (e.g., always giving the LLM text+image) often introduces noise rather than signal.
Figure 1: (a) Mario significantly improves cross-modal similarity. (b) The Venn diagram proves that ~30% of nodes require a specific modality configuration to be correctly classified.
Methodology: The Two-Stage "Mario" Framework
Stage 1: Graph-Conditioned Alignment (GVLM)
Instead of using a frozen CLIP, Mario uses a Graph-conditioned Vision-Language Model. It embeds a Transformer-based "Mixer" into the encoding process. This allows a node's visual representation to "talk" to the textual representations of its neighbors before alignment. By the time contrastive learning (InfoNCE) happens, the embeddings are already aware of the graph's structural context.
Stage 2: Modality-Adaptive Instruction Tuning
This is where the magic happens. Mario doesn't force a single template. It builds a Bank of Templates (Text-Only, Image-Only, Multimodal).
A lightweight Modality-Adaptive Prompt Router (MAPR) looks at the node's features and its neighborhood to decide which template is best. During training, the LLM provides feedback (via loss) to the router, teaching it which modality configuration leads to the most accurate predictions.
Figure 2: The dual-stage pipeline showing Stage 1 (Alignment) and Stage 2 (Router-led Instruction Tuning).
Experimental Battleground: Crushing the Baselines
Mario was tested on diverse domains including Amazon (E-commerce), Reddit (Social), and Goodreads (Literature).
1. Superior Reasoning Power
In standard supervised settings (Single-Focus), Mario outperformed leading baselines like LLaVA-1.5, Qwen2.5-VL, and MLaGA. Specifically, on the "CDs" dataset, it improved accuracy by nearly 7% absolute over the best existing multimodal graph models.
2. Zero-Shot Generalization
Perhaps the most impressive feat is Mario's performance on unseen graphs. When trained on "Toys" and tested on "Movies", Mario achieved 41% accuracy, while most GNN-based methods struggled to stay above 10%. This highlights that the "routing logic" learned by Mario captures fundamental reasoning patterns that transfer across domains.
3. Convergence Efficiency
Despite having more complex routing, Mario converges 1.3x to 2.3x faster than fixed-template models. By selecting the right modality, the LLM avoids wasting gradients on noisy or irrelevant inputs.
Figure 3: Mario (Adaptive) converges significantly faster and to a lower loss than fixed-modality templates.
Critical Insight: The Homophily of Preference
One of the paper's most fascinating findings is that modality preference follows a homophily pattern. Just as connected nodes often share labels (label homophily), they also tend to prefer the same modalities. For instance, in a cluster of "Art Supplies," both users and the model might find visual features consistently more useful across the entire neighborhood.
Conclusion & Future Look
Mario proves that LLMs can be powerful graph reasoners if—and only if—we respect the structural nature of the data. The move from "captions" to "graph-aware tokens" and from "one-size-fits-all prompts" to "adaptive routing" represents a significant shift in how we handle heterogeneous multimodal data. Future work could likely extend this to temporal graphs or dynamic social streams where modality utility shifts over time.