Decoding Sentiment: A Comparative Benchmark of Machine Learning for Twitter Analytics
17392_Sentiment analysis of foreign tourists to Bangkok using data mining through online social network.
This paper presents a comparative analysis of machine learning algorithms for sentiment classification on Twitter data. It evaluates four core methods—Decision Trees, Naïve Bayes, Support Vector Machines (SVM), and Artificial Neural Networks (ANN)—using a dataset of tweets related to Bangkok to categorize public sentiment into positive, negative, and neutral classes.
TL;DR
This research evaluates four foundational machine learning techniques—Decision Trees, Naïve Bayes, SVM, and Artificial Neural Networks (ANN)—to classify the sentiment of tweets regarding Bangkok. By utilizing techniques like TF-IDF and Information Gain, the study identifies Artificial Neural Networks as the most effective baseline, reaching an accuracy of 80.33%, while highlighting the significant limitations of the Naïve Bayes approach in real-world linguistic environments.
Problem & Motivation
Twitter represents a "digital pulse" of humanity, yet its data is notoriously difficult to process. The core challenge lies in the high dimensionality of language and the noise (hashtags, slang, typos). Traditional sentiment analysis often fails to strike a balance between computational efficiency and the ability to capture nuanced meaning. The authors aim to benchmark which classical machine learning architectures provide the best "bang for the buck" for urban sentiment monitoring.
Methodology: The Core
The workflow follows a rigorous text-mining pipeline:
- Feature Extraction: Utilizing TF-IDF (Term Frequency-Inverse Document Frequency) to quantify the importance of words relative to the entire corpus.
- Feature Selection: Applying Information Gain (IG) based on entropy to reduce the feature space, ensuring only the most discriminative words are used for training.
- Classification: The study compares the "probabilistic" (Naïve Bayes), the "logic-based" (Decision Tree), and the "geometric/connectionist" (SVM and ANN) approaches.
The mathematical backbone of the ANN approach relies on weight adjustments between layers:
The function f represents the activation layer that transforms input vectors into sentiment class probabilities.
Experiments & Results
The researchers conducted extensive testing using a confusion matrix to track True Positives (T), False Positives (F), and specific class overlaps (Neutral, Positive, Negative).
Performance Leaderboard:
| Technique | Accuracy (%) |
|---|---|
| Artificial Neural Network (ANN) | 80.33 |
| Support Vector Machine (SVM) | 80.11 |
| Decision Tree | 79.83 |
| Naïve Bayes | 55.66 |
The drastic failure of Naïve Bayes (55.66%) is a critical finding. It suggests that the "independence assumption" (treating every word as if it has no relation to the next) is too reductive for the complex structure of social media dialogue.
Detailed Confusion Matrix showing the distribution of predictions across classes.
Critical Analysis & Conclusion
Takeaway
The success of the Artificial Neural Network and SVM models highlights that sentiment analysis is a non-linear problem. The ability of these models to map text into a higher-dimensional latent space allows them to separate "Neutral" and "Positive" sentiments much more effectively than simpler logic trees.
Limitations & Future Work
While 80% accuracy is a strong baseline for classical ML, current SOTA (State of the Art) work in LLMs (Large Language Models) often exceeds 90% in similar tasks. This study serves as a vital benchmark for "lightweight" deployments where massive GPUs are not available, but future work should explore Ensemble Methods—combining ANN and SVM to create a more robust "voting" classifier that can mitigate individual model biases.