LAAN: Bridging Sentiment Lexicons and Deep Learning via Linguistic-Aware Attention

TL;DR

The Linguistic-Aware Attention Network (LAAN) is a surgical refinement of CNNs for sentiment analysis. It moves beyond "black-box" feature extraction by explicitly modeling the relationship between general context and a predefined sentiment lexicon. By implementing an interactive attention at the word level and a dynamic attention at the phrase level, it achieves SOTA results on MR and SST datasets.

Problem & Motivation: The "Forgotten" Lexicon

In the rush toward end-to-end deep learning, researchers have often sidelined traditional sentiment lexicons—lists of words with known polarities (e.g., "excellent" vs. "disappointing"). While models like BiLSTM and CNNs learn great representations, they often lack the explicit inductive bias needed to focus on the truly emotive parts of a sentence.

The authors argue that sentiment analysis shouldn't just be about context; it's about how context modifies sentiment words. Existing "linguistic regularization" methods were a step in the right direction but lacked the granularity to capture dynamic phrase-level interactions.

Methodology: The Two-Stage Strategy

LAAN's architecture is built on the philosophy that sentiment is hierarchical.

1. Word-Level Interactive Attention

Instead of treating all words equally, LAAN splits the input into Context Words ($W^c$) and Sentiment Words ($W^s$) based on a lexicon.

• Correlation Matrix ($C$): The model computes $C=W^c\cdot (W^s{)}^T$ to find the "mutual information" between every word pair.
• Mutual Enhancement: It generates weight vectors $\alpha$ and $\beta$ to highlight context words relevant to sentiment, and sentiment words relevant to the global context. This creates "sentiment-enhanced" embeddings.

2. Phrase-Level Dynamic Semantic Attention

After the word-level stage, the model applies multi-gram convolutions (varying window sizes) to extract local features.

• Filtering Phrase Chunks: Not all N-grams are created equal. The Dynamic Semantic Attention mechanism acts as a sophisticated filter, identifying which phrase chunks (e.g., "not very good" vs. "good") carry the most weight for final polarity prediction.

(Note: Above image represents the visual schematic of the network as presented in the original poster/paper.)

Experiments & Results: SOTA Performance

LAAN was tested against traditional RNNs, LSTMs, and the standard CNN for sentence classification.

The results reveal two critical insights:

1. Linguistic Information Matters: Models that used lexicons (LR-LSTM and LAAN) consistently beat "pure" neural models.
2. Structural Attention > Regularization: LAAN’s method of attending to linguistic cues was significantly more effective than the "Regularization" (LR) methods which only used linguistic rules as auxiliary loss constraints.

Critical Analysis & Conclusion

Takeaway

LAAN proves that the "old" world of sentiment lexicons and the "new" world of deep attention-based neural networks are not mutually exclusive. By designing architecture that reflects the Linguistic Structure of the task, we can achieve better performance with potentially less data.

Limitations & Future Work

• Lexicon Dependency: The model's performance is still tethered to the quality and coverage of the external sentiment lexicon.
• Contextual Sarcasm: While dynamic attention helps, extremely nuanced sarcasm that reverses polarity might still require deeper transformer-based contextual modeling (which was in its infancy during this paper's publication in 2018).
• Modern Pivot: To take this further today, one could integrate these linguistic "priors" into the attention heads of a pre-trained Transformer (like BERT or RoBERTa).