Who Likes What and Why? Bridging the Gap Between Visual Semantics and Human Personality
‘Who Likes What and, Why?’ Insights into Modeling Users’ Personality Based on Image ‘Likes’
This paper introduces a semantic-centered framework for modeling user personality traits (Big Five) based on implicit image "likes" on Flickr. By proposing a novel Feature Selection based Ordinal Regression (FS-OR) method and a two-stage F2A+A2P (Features-to-Answers to Personality) approach, the authors achieve a 12-15% improvement over the state-of-the-art in personality recognition.
TL;DR
Understanding why a user "likes" a specific photo is a holy grail for personalized recommendation. This research moves beyond raw pixels to model the Big Five personality traits (Openness, Conscientiousness, Extraversion, Agreeableness, Neuroticism) using a unique two-stage approach. By predicting how a user would answer a personality test based on their "likes," and then calculating their traits, the authors achieved a 15% performance boost over traditional methods.
Contextual Positioning
In the landscape of "Personality Computing," this work sits at the intersection of Computer Vision and Psychometrics. While most models follow a "Features-to-Personality" (F2P) black-box approach, this study champions the F2A+A2P (Features-to-Answers + Answers-to-Personality) pipeline. It treats personality not as a single regression target, but as a structured construct derived from specific behavioral cues.
The Core Challenge: The "Multimedia Unconscious"
Why is predicting self-assessed personality so hard?
- Abstract Mapping: Mapping "edge statistics" to "Neuroticism" is mathematically valid but psychologically weak.
- Ordinal Nature: Survey answers (1 to 5) are not just numbers; they represent an ordered intensity that standard regression often ignores.
- Low-Level Limits: Previous SOTA relied on aesthetics (rule of thirds, contrast). However, personality is often reflected in what is in the image (semantics) rather than just how it was shot.
Methodology: High-Level Semantics + FS-OR
The authors propose a robust three-pillar feature set: People (face counts, gender, poses), Content (clutter, saliency, natural vs. graphics), and Affect (color names, emotional statistics).
The FS-OR Algorithm
To process these features, they developed Feature Selection based Ordinal Regression (FS-OR).
- Ordinal Transformation: It converts the 1-5 rating scale into a series of binary classification tasks while maintaining the order (e.g., a score of 4 must be greater than 3).
- Sparse Selection: Not every image feature matters for every trait. FS-OR uses a cutting-plane algorithm to find the specific "violated" features that contribute most to specific BFI-10 questions.
Experimental Insights: Who Likes What?
The study provides fascinating psychological correlations validated by the model:
- Extraversion: High affinity for images with many people, specifically the opposite gender.
- Neuroticism: Prefers "cluttered" images and computer graphics over simple, clean natural scenes.
- Conscientiousness: Shows a preference for focused, planned shots and black-and-white photography (symbolizing focus and subtlety).
Quantitative SOTA Break-through
The F2A+A2P approach significantly outperformed direct F2P mapping across all traits.
As shown in the results, Extraversion (E) and Agreeableness (A) saw the highest classification accuracy gains, reaching 84.7% and 87.5% respectively, thanks to the inclusion of "People" based semantic features.
Critical Analysis & Future Outlook
Strengths: This work bridges the gap between signal processing and social science. The move from F2P to F2A+A2P provides a "white-box" glimpse into why the model thinks a user has a certain trait.
Limitations:
- Dataset Bias: The PsychoFlickr dataset consists of "Pro" users, potentially skewing results toward artistic preferences (Openness).
- Implicit Feedback: The model only uses "likes." The absence of "dislikes" makes it harder to map the full boundaries of a user's personality.
Conclusion: This research proves that our digital footprints (image likes) are rich with latent psychological data. For developers of recommender systems, the message is clear: to solve the cold-start problem, look at the semantic "why" behind a user's preference. Personality-aware AI is no longer a psychological theory; it is a quantifiable engineering reality.