RNN-Boost: Fusing Social Media Wisdom and Adaptive Boosting for Stock Prediction

TL;DR

Predicting stock volatility has long been the "Holy Grail" of FinTech. This paper introduces RNN-boost, a hybrid architecture that scrapes news from Sina Weibo (China's Twitter) and processes it through a Gated Recurrent Unit (GRU) network enhanced by Adaboost. By looking beyond simple sentiment to include LDA topic features, the model achieves a remarkable 70.17% accuracy in predicting the direction of the HS300 index.

Problem & Motivation: Beyond the Random Walk

Traditional finance theories like the Efficient Market Hypothesis (EMH) suggest that making consistent profits by predicting the market is nearly impossible. However, the rise of Online Social Networks (OSN) has changed the game. Unlike traditional media, OSN news is succinct, spreads instantly, and reflects the "public mood" in real-time.

The authors identify two major gaps in prior work:

1. Feature Scarcity: Most models only use technical prices or basic "positive/negative" sentiment.
2. Model Instability: Single neural networks are highly sensitive to initial parameters, leading to inconsistent results that are risky for actual trading.

Methodology - The Core

The paper's "Secret Sauce" lies in its two-pronged approach: multidimensional feature engineering and the RNN-boost training loop.

1. Content Features: Sentiment + LDA

The authors don't just ask if the news is good or bad; they ask what the news is about.

• Sentiment Analysis: A Naive Bayes-based approach assigns a "positivity" score to daily Weibo posts.
• Latent Dirichlet Allocation (LDA): This discovers hidden thematic structures (topics). For instance, news about "regulations" might impact the market differently than news about "consumer humor," even if both have neutral sentiment.

2. The RNN-Boost Architecture

Instead of relying on one "hero" model, the authors use an ensemble.

• Base Learner: A 2-hidden-layer RNN with GRU units to capture long-term dependencies in time-series data.
• Boosting Loop: Using the Adaboost.R2 logic, the system trains the first RNN, identifies the dates where the prediction error was highest, and then forces the next RNN to focus on those "difficult" dates by increasing their sample weight.
• Aggregated Output: The final prediction isn't a simple average; it’s a weighted median, where the most "confident" (low-error) models have a louder voice.

Fig 1: The RNN-Boost workflow, from data collection to weighted median output.

Experiments & Results

The model was tested on the Shanghai-Shenzhen 300 (HS300) index using data from 2015 to 2017.

The Impact of Social Media

The experiment clearly showed that "Context is King."

• Technical features only: ~53% accuracy (barely better than a coin flip).
• Tech + Sentiment: ~64% accuracy.
• Tech + Sentiment + LDA: ~65% to 70% accuracy.

Boosting vs. Single RNN

The boosting mechanism didn't just marginally increase the top-end accuracy; it significantly raised the "floor". While a single RNN might occasionally perform poorly due to bad initialization (61% min accuracy), the RNN-boost reached a minimum "worst-case" accuracy of 64.06%, proving its robustness for real-world deployment.

Table 1: RNN-Boost vs. SVR, MLP, and other baselines.

Critical Analysis & Conclusion

Takeaway

The primary insight is that financial markets are social constructs. By capturing the "latent" topics of conversation via LDA, the model accounts for the nuance that simple sentiment analysis misses. Furthermore, the use of Adaboost transforms RNNs from volatile predictors into a stable, reliable ensemble.

Limitations

Despite the high accuracy, the paper utilizes relatively simple technical indicators. Modern quantitative trading often uses hundreds of "alphas" (signals). Additionally, the LDA model is static; using dynamic topic modeling could further capture how popular themes shift over time.

Future Work

The authors suggest that future iterations will focus on automated feature selection to filter the most potent technical rules (like Moving Average or Relative Strength) into the RNN-boost framework, potentially pushing the accuracy barrier even further.