Attention-based LSTM: Decoding Human Personality through Topic and Sentiment

TL;DR

Predicting human personality from social media text is a complex task requiring models to understand not just words, but the intent and emotion behind them. This paper introduces an Attention-based LSTM model that leverages LDA for topic modeling and Word2Vec for sentiment analysis. By weighting user preferences and emotional cues, the model achieves a significant performance boost in identifying the "Big Five" personality traits compared to traditional machine learning baselines.

Background & Motivation: The Digital Mirror

In an era where our digital footprints define us, social networking sites have become a goldmine for psychological profiling. However, prior research often treated text as a "bag of words" or relied on shallow linguistic features (like counting pronouns). The authors argue that personality manifests in the intersection of Theme (what we care about) and Affect (how we react). A person high in Extraversion might frequently discuss social gatherings with positive sentiment, while someone high in Neuroticism might use negative affect words in the context of daily stress.

Methodology: The Architecture of Personality

The core of the paper lies in its hybrid architecture, which moves beyond simple sequence modeling to "Attentive Prediction."

1. The Multi-Channel Input

The model processes raw text via two primary channels:

• Topical Quantification (LDA): Users' interests are clustered into 30 distinct topics. This provides a "thematic preference" vector that acts as a prior for the user's focus.
• Semantic Vectorization: Using the Skip-gram method (Word2Vec), text is converted into high-dimensional vectors that preserve sentiment proximity.

2. The Recurrent Engine with Attention

While a standard LSTM can process text sequences, it often loses focus on the most "telling" words. The authors introduce an Attention Layer that combines the LSTM output matrix $H$ with the topic distribution $\alpha$.

The attention mechanism calculates a weight matrix to highlight specific features: $r=H\otimes \alpha$ This ensures that when the model assesses personality, it "pays attention" to words that align with the user's identified interests and emotional peaks.

Experiments & Results: Slaying the Baselines

The researchers conducted extensive ablation studies to find the "sweet spot" for deep learning parameters. They discovered that an LSTM with 2 hidden layers and 128 neurons per layer achieved the highest precision without incurring the exponential time costs associated with deeper networks.

Performance Comparison

When stacked against traditional models like SVM, Random Forest, and Bayesian Networks, the proposed model showed clear superiority:

The Jump in Recall (+6.47%) and F1 (+5.02%) indicates that the model is far more robust at identifying true personality traits across the noise of casual social media posts.

Critical Insight & Outlook

The success of this model stems from its Inductive Bias: it assumes that personality is encoded in the consistency of topic-sentiment pairs. By using LDA as a "top-down" guide and LSTM for "bottom-up" sequence learning, the authors bridge the gap between traditional psychology and modern NLP.

Limitations:

• The precision (~58%) suggests that text alone, while powerful, may not be enough for clinical-grade profiling.
• The use of LSTM, while efficient, may soon be eclipsed by Transformer-based models (like BERT) which utilize self-attention even more effectively.

Future Work: Integrating cross-platform data (e.g., matching a user's LinkedIn "Conscientiousness" with their Facebook "Extraversion") remains the frontier for this technology.