[NeurIPS 2025] NOBEL: Unifying the Brain's "Electric" and "Metabolic" Tongues with LLMs

TL;DR

The human brain speaks multiple "languages": the rapid electrical pulses of neurons (EEG/MEG) and the slow, oxygen-rich blood flow of metabolism (fMRI). Historically, AI models have only mastered one at a time. NOBEL (Neuro-Omni-modal Brain-Encoding LLM) changes the game by becoming the first foundation model to unify all major non-invasive brain modalities into a single semantic workspace. By leveraging the reasoning power of LLMs, it doesn't just decode brain signals; it interprets them in the context of the world we see and hear.

The Fragmentation Problem: A Tale of Two Resolutions

In neuroscience, we have a trade-off often called the "frequency gap":

1. EEG/MEG: Millisecond-level speed, but "blurry" spatial location.
2. fMRI: Pinpoint spatial accuracy, but a 5-10 second "lag" due to blood flow dynamics.

Because these signals look so different, researchers built isolated silos. An EEG model couldn't understand an fMRI scan. Moreover, these models were "stimulus-blind"—they ignored the very images or sounds that triggered the brain activity, missing the ground truth required for deep semantic alignment.

Methodology: Building a Universal Translator

NOBEL treats brain decoding as a multimodal alignment problem.

1. The Dual-Path fMRI Strategy

The authors realized that fMRI data contains two things: who you are (static physiology) and what you are looking at (dynamic perception).

• Path A (Raw Voxels): Uses NeuroSTORM to capture the global "landscape" of the brain.
• Path B (Beta Weights): Uses a MindEye2-inspired structure to extract the semantic "content" of a specific thought or stimulus.

2. The Unified Electromagnetic Encoder

Instead of treating EEG and MEG as different entities, NOBEL uses BrainOmni to map them into a shared latent space, acknowledging their common biophysical origin.

3. The LLM as the "Semantic Glue"

All these signals—alongside text prompts and actual audio-visual stimuli—are projected into the embedding space of Qwen2.5-Omni-7B. This allows the model to answer complex questions like: "Is this subject currently seeing a cat or a dog?" or "Describe the emotion in this EEG segment."

Figure 1: The NOBEL architecture, showing how diverse modalities are tokenized and fed into the LLM backbone.

Experimental Breakthroughs

The power of NOBEL is most evident when it combines modalities. In the Nat-View dataset, where EEG alone performed poorly (16.7%), the synergistic NOBEL model reached 92.8% by filling in EEG's temporal gaps with fMRI's spatial clarity.

Brain-to-Text Decoding

NOBEL doesn't just classify; it describes. On the Natural Scenes Dataset (NSD), it generates image captions directly from brain activity that closely match the actual visual stimuli presented to the subjects.

Figure 2: Qualitative examples of NOBEL predicting image descriptions directly from fMRI beta weights.

Cross-Modal Verification

One of the most unique tasks NOBEL performs is Stimulus-Brain Verification. Given both a brain signal and an image, the model can verify if that specific image was the cause of the neural response (achieving 92.7% accuracy on NSD). This "causal" reasoning is a leap forward from simple pattern matching.

Critical Analysis & Future Outlook

NOBEL proves that the "One Brain, One Model" philosophy is viable. By using an LLM as a backbone, the model inherits a vast "common sense" understanding of the world, which it uses to make sense of noisy neural data.

Limitations:

• Data Scarcity: While NOBEL uses many datasets, "paired" data (simultaneous EEG + fMRI) is still rare and difficult to collect.
• Computational Cost: Running a 7B parameter model for real-time BCI (Brain-Computer Interface) applications remains a challenge for edge devices.

The Takeaway: NOBEL is a bridge. It connects the "lightning" of electrical neural firing with the "tide" of metabolic flow, all under the umbrella of human language. This paves the way for truly universal brain-AI interfaces that don't care which sensor you're wearing—they just understand the mind.