EBoW: Strengthening Cyberbullying Detection through Embedding-Enhanced Features
Automatic detection of cyberbullying on social networks based on bullying features
The paper introduces EBoW (Embeddings-enhanced Bag-of-Words), a novel representation learning framework for cyberbullying detection. By combining traditional BoW and Latent Semantic Analysis (LSA) with embedding-expanded "bullying features," the model achieves SOTA performance on Twitter datasets using a linear SVM classifier.
TL;DR
With the rise of social media, cyberbullying has become a critical public health issue. Traditional NLP models often treat bullying detection as a generic text classification task, missing the nuance of offensive language. This paper introduces EBoW (Embeddings-enhanced Bag-of-Words), which systematically expands a set of "insulting seeds" using word embeddings to capture the semantic variety of harassment, leading to superior detection performance on platforms like Twitter.
Problem & Motivation: Beyond Keyword Matching
Why is cyberbullying hard to detect? Unlike physical bullying, it is persistent (24/7) and often uses coded language, slang, or synonyms that simple keyword filters miss.
Previous works attempted to solve this by:
- Standard BoW: Treating "idiot" and "moron" as unrelated dimensions.
- Manual Weighting: Arbitrarily doubling the weight of a few swear words, which lacks a theoretical basis and fails to adapt to new slang.
The authors identified that the Inductive Bias of the model must include the semantic proximity of offensive terms. If "slut" is a known bullying term, semantically similar words retrieved via latent space (like "whore" or "hypocrite") should also be treated as high-signal features.
Methodology: The EBoW Framework
The core innovation is the Representation Learning phase. Instead of relying on a single feature type, the authors concatenate three distinct perspectives:
- Bag-of-Words (BoW): Captures the raw presence of unigrams and bigrams using TF-IDF.
- Latent Semantic Analysis (LSA): Reduces dimensionality to capture global context and reduce noise.
- Bullying Features (The Secret Sauce):
- Start with 350 "insulting seeds" (e.g., nigga, bitch, fuck).
- Use a Word2vec model (trained on 400M tweets) to find the top- most similar words for each seed.
- Assign weights based on Cosine Similarity. For a bigram like "stupid jerk," they use an additive model: .
Figure 1: The EBoW feature concatenation process leading to a Linear SVM classifier.
Experimental Insights
The researchers tested EBoW against strong baselines (LDA, LSA, and scaled BoW) on a labeled Twitter dataset.
Key Findings:
- Performance Lift: EBoW achieved the highest Precision (76.8), Recall (79.4), and F1-Score (78.0).
- The "Goldilocks" Zone: The parameter (number of expanded words) is crucial. Too small (), and the model lacks coverage; too large (), and noisy, non-offensive words dilute the feature space.
| Method | Precision | Recall | F1 Score |
|---|---|---|---|
| BoW | 75.6 | 77.8 | 76.6 |
| sBoW (Scaled) | 75.7 | 78.3 | 76.9 |
| EBoW (Ours) | 76.8 | 79.4 | 78.0 |
Figure 2: Sensitivity analysis of parameter h showing the peak performance around h=50.
Critical Analysis & Conclusion
The beauty of EBoW lies in its Interpretability. Unlike deep black-box neural networks, we can see exactly which expanded words (like those in Figure 2's word cloud) are driving the classification.
Limitations:
- The model still relies on a "seed list," which may require periodic updates as Internet slang evolves (e.g., "leetspeak" or memes).
- The additive model for bigram embeddings is a simplification that might not capture complex sarcasm.
Takeaway: This work demonstrates that effectively "grounding" embeddings with domain-specific lexicons is a powerful way to augment classical machine learning pipelines, providing a robust middle ground between pure manual engineering and full deep learning.