Detecting Stadium Violence via Instagram: A Multimodal Emotional Analysis
An Exploratory Sentiment and Facial Expressions Analysis of Data from Photo-sharing on Social Media: The Case of Football Violence
The paper explores an automated approach to predicting football-related violence by performing sentiment and facial expression analysis on Instagram data. Using commercial APIs like Microsoft Oxford and SightCorp, the researchers analyzed photos and comments from 2015 matches to distinguish between peaceful games and those involving crowd violence.
TL;DR
Can a selfie predict a riot? This paper explores the "digital pulse" of football fans by analyzing Instagram data. By extracting facial expressions and text sentiment from photos geofenced to specific stadiums, researchers found that anger levels are significantly higher (2.4x) during matches that lead to violence. They propose a classification system that could theoretically act as an early warning system for law enforcement.
Motivation: The Blind Spot of Physical Policing
Despite heavy police presence at high-stakes football matches, violence remains a persistent issue. The core problem is that police react to actions, not intentions. The authors argue that the "prevailing mood" of a crowd—captured in real-time on social media—is a proxy for violent intent. Since fans constantly upload photos to Instagram, this data provides a continuous stream of the crowd's emotional state.
Methodology: From Selfie to Security Insight
The researchers focused on eight specific "violent" matches from 2015 and compared them to "normal" home games. Their pipeline involved two primary data streams:
- Facial Emotion Recognition (FER): Utilizing the Microsoft Oxford Project (now Azure Face API) and SightCorp InSight, they processed faces in photos to quantify emotions like anger, contempt, and fear.
- Sentiment Analysis: Text comments were lemmatized and processed using Support Vector Machines (SVM) to identify offensive language or threats.
Figure: Example of facial emotion recognition used to categorize crowd sentiment.
The Feature Set
The authors didn't just look at average emotions; they used SelectKBest to find the most predictive features, which included:
- Ratio of photos with anger levels > 70%.
- Maximum "surprise" levels.
- Median "fear" levels.
- Social engagement metrics (average likes).
Experimental Results: The "Anger" Signal
The data revealed a stark contrast. While most photos at any match are "happy" or "neutral" (the "stadium selfie" bias), the anger channel provided the most significant delta.
Figure: The distribution of anger was significantly skewed toward higher scores in matches where fights occurred.
Key Stats:
- Anger Levels: 2.4x higher in violent matches (via MS Oxford).
- Classification Accuracy: The Naive Bayes classifier achieved 83.3% accuracy in distinguishing between match types using only five features.
Table: Comparison of different classifier performances (Naive Bayes, KNN, Decision Trees) across various feature counts.
Critical Insights & Limitations
While the results are promising, the authors acknowledge several "noise" factors:
- The "Happy" Filter: People tend to post happy photos even if the atmosphere is tense, which can mask the true sentiment (Social Media Bias).
- Data Scarcity: The study was limited to a small selection of matches where metadata was sufficient.
- Context Matters: Factors like alcohol consumption, the final score, and long-standing team rivalries were not fully integrated into the model.
Conclusion and Future Work
This paper serves as a "proof of concept" for sentiment-based security. The shift from manual monitoring to automated emotional tracking represents a significant leap in predictive policing. The authors suggest that moving toward real-time video processing (rather than static photos) will be the next frontier in understanding the dynamic emotional shifts of a crowd before the first punch is thrown.