Beyond the Beat: Decoding Music Preference through Personality and Physiology

TL;DR

This study explores a multimodal approach to Music Recommendation Systems (MRS) by combining what we listen to (acoustic features) with who we are (personality) and how our bodies react (physiological signals). By analyzing 628 listening records, the research demonstrates that while acoustic data is foundational, incorporating personality traits significantly sharpens recommendation accuracy, with physiological data offering untapped potential for real-time context.

The "User-Centered" Gap in MIR

In the field of Music Information Retrieval (MIR), most systems are excellent at identifying "songs that sound like X." However, they often fail to understand the listener. Why does a person love a specific track today but skip it tomorrow? The missing link is the User Context and User Properties.

Prior work has looked at personality in isolation or used physiological signals primarily for emotion recognition. This paper bridges the gap by testing if these diverse data streams can be fused into a single predictive model for better recommendations.

Methodology: Fusing Content, Traits, and Biometrics

The researchers conducted a user experiment involving 23 participants using an in-house system called "Moody." They gathered three distinct feature sets:

1. Acoustic Features (231 dim): Timbre, rhythm, pitch, and harmony extracted via LibROSA.
2. Personality Features (5 dim): The "Big Five" traits (Openness, Conscientiousness, Extroversion, Agreeableness, Emotional Stability).
3. Physiological Signals (61 dim): Heart Rate (HR), Electrodermal Activity (EDA), and Skin Temperature (TEMP) recorded via an Empatica E4 wristband.

The target variable was an implicit rating (0-4) derived from behavior: a full play counted higher than an early skip.

Table: The three feature groups used to test the incremental value of multimodal data.

Key Results: Personality Reigns Supreme

The experiment compared four regression models: SVR, Decision Tree (DT), Neural Networks, and LightGBM.

• The Winner: The Decision Tree model using all features (Group 3) achieved an RMSE of 0.718, significantly outperforming models without personality data.
• The Power of Personality: Including personality traits led to a statistically significant improvement in prediction.
• The Physiological Surprise: While the aggregate group of physiological features didn't show a massive statistical leap for the whole dataset, individual features were highly influential.

Table: Regression model performance across different feature groups.

Deep Insight: Which Features Matter Most?

The feature importance analysis (Figure 1 in the paper) reveals a fascinating hierarchy. The #1 predictor was TEMP_mean (mean skin temperature), followed by spectral contrast (acoustic) and Extroversion (personality).

Out of the top 10 features, 4 were physiological and 3 were personality-based. This suggests that while "physiological signals" as a broad category might be noisy, specific biometrics are incredibly potent indicators of whether a user is vibing with a track.

Figure 1: Normalized information gain showing the dominance of temperature and extroversion.

Critical Analysis & Future Outlook

Takeaway: If you want to build a better recommender today, ask the user a 10-item personality quiz. It provides the "Inductive Bias" necessary to understand long-term preferences.

Limitations: The sample size (23 participants) is small. Physiological signals are also notoriously "noisy" and can be affected by ambient temperature or physical activity, not just the music.

The Future: As wearables become ubiquitous, MRS will likely move toward "Affective Computing," where the system adjusts the queue based on your heart rate variability (HRV) or stress levels. This paper provides a foundational stepping stone toward that empathetic, multimodal digital library.