Twitter Sentiment vs. Reality: Can Social Media Track Crisis Approval Ratings?

TL;DR

In political science, the "rally around the flag" effect describes the sudden, temporary spike in a leader's approval during national crises. This paper explores if we can use Twitter to monitor this phenomenon in real-time. By training an LSTM on 4 years of Trump-era tweets, the researchers found that while social media sentiment fluctuates, it lacks the speed and demographic depth of professional polling, making it an unreliable tool for crisis situational awareness.

Problem & Motivation: The Latency of Polling

When a crisis like COVID-19 hits, leaders need immediate feedback on public trust to manage policy effectively. Traditional polling (solicited surveys) is the gold standard but acts as a "rear-view mirror"—it’s slow and expensive.

The researchers identified two major gaps in previous attempts to use Twitter as a substitute:

1. Technological Gap: Most prior work used simpler models like Naive Bayes, which fail to capture the nuances of linguistic sequences.
2. Contextual Gap: General sentiment analysis often confuses "negativity about a situation" with "negativity toward a specific leader."

Methodology: Directed Sentiment & LSTM

The core innovation lies in the transition from general sentiment to directed sentiment. Instead of just labeling a tweet as "negative," the authors manually labeled tweets based on whether the negativity was directed at the President or his opponents.

The Model

They chose a Long Short-Term Memory (LSTM) network. Unlike standard feed-forward networks, LSTMs have feedback connections, allow them to process sequences of data—vital for understanding the flow and context of 280-character political discourse.

Table 1: The LSTM (not shown in this specific table but discussed in text) outperformed the AutoML and MNB baselines shown here, achieving 69% overall accuracy.

Experiments & Results: A Reality Check

The study compared Twitter sentiment against the FiveThirtyEight aggregate poll data from February to May 2020.

Key Findings:

• The "Rally" Mismatch: In late March, Trump’s official approval rating shot up rapidly. On Twitter, however, the sentiment shift was a "slow creep" rather than a sharp spike.
• Weak Correlation: The correlation between positive tweets and actual approval ratings never exceeded 0.25. In data science terms, this is a very weak signal.
• Demographic Distortion: The authors attribute this failure to the "Twitter bubble." Users are younger, more urban, and more male than the general US population, creating an echo chamber that doesn't reflect the national "pulse."

Figure 3: While real-world approval (blue) and disapproval (orange) crossed paths sharply in March, Twitter sentiment failed to mirror this volatility.

Deep Insight & Conclusion

The study provides a sobering conclusion for digital strategists: Twitter is not the nation.

While the researchers improved the technical side by using LSTMs and directed sentiment, the inherent selection bias of the platform remains an insurmountable hurdle for now. For leaders seeking "situational awareness" during a crisis, social media provides a noisy, distorted reflection rather than a clear window into the public mind.

Future Outlook

The authors suggest that future work should focus on Aspect-Level Sentiment Classification—understanding not just if people are angry, but what specific policy or action triggered the anger. Additionally, moving beyond Twitter to more demographically diverse platforms might bridge the gap between digital "noise" and national "signal."