GAT-NSR: Synergy of Multi-Head Attention and Neural Collaborative Filtering for Social Recommendation

TL;DR

In the era of information overload, social recommendation uses "social trust" to solve data sparsity. GAT-NSR is a novel framework that captures high-quality user/item embeddings using Multi-Head Attention and processes them through a Neural Collaborative Recommendation module. By fusing social graphs with interaction graphs and combining deep learning with traditional inner products, it sets a new benchmark for rating prediction.

Problem & Motivation: Beyond Matrix Factorization

Standard Collaborative Filtering (CF) relies on the user-item interaction matrix. However, when data is sparse, these models fail. Social recommendation mitigates this by incorporating social relationships ("birds of a feather flock together").

The authors identify two critical gaps in existing state-of-the-art (SOTA) methods:

1. Representation Precision: Most methods don't capture features from different representational subspaces.
2. Interaction Complexity: Simple inner products used in Matrix Factorization cannot capture the "intrinsic and complex" behaviors of modern web users.

Methodology: The Architecture of GAT-NSR

GAT-NSR's architecture is divided into three distinct stages: Initial Embedding, Latent Vector Learning, and Neural Collaborative Recommendation.

1. Multi-Head Attention for Message Passing

The model processes two graphs: the Social Graph (User-User) and the Interaction Graph (User-Item). For a user $u_i$, the latent vector is a fusion of:

• Item-based representation: Aggregating items $v_j$ and rating scores $e_r$.
• Social-based representation: Aggregating features of connected friends $u_j$.

By using Multi-Head Attention, the model can attend to different neighbors differently across $K$ heads, preventing the "smoothing" effect of standard GCNs.

Note: The architecture illustrates the dual-stream feature extraction from social and interaction graphs.

2. Neural Collaborative Recommendation (NCR)

Instead of just feeding embeddings into a dot product, GAT-NSR uses a hybrid approach:

• Collaboration Layer: It concatenates $h_i$, $h_j$, and the element-wise product $h_i\odot h_j$. The element-wise product acts as a "shallow" feature that preserves classic CF intuition.
• Neural CF Layers: A Multi-Layer Perceptron (MLP) follows, providing the non-linear flexibility needed to model deep interactions.

Experiments & Results

The model was tested on FilmTrust and Epinions. GAT-NSR consistently achieved the lowest MAE and RMSE compared to 8 baselines, including GraphRec and NeuMF.

Key Insights:

• Optimizing Attention Heads: The authors found that increasing the number of heads ($K$) improves performance up to a point ($K=4$), after which the reduced dimensionality per head ($d/K$) leads to information loss.
• The Power of Hybrid Layers: Ablation studies (NCR vs. NCR-IP) showed that keeping the inner product part ($h_i\odot h_j$) alongside the MLP yields better results than using MLP alone. This suggests that "shallow" inductive biases are still valuable.

Critical Analysis & Conclusion

GAT-NSR proves that social recommendation is not just about what data you have, but how you aggregate it. By treating social and interaction data as two perspectives of the same user, and then applying a hybrid deep-shallow predictor, the model achieves superior robustness against data sparsity.

Future Outlook: The authors suggest incorporating side information (user attributes/item categories) and moving towards Temporal/Dynamic Graphs to capture the evolving nature of social influence. For practitioners, this paper highlights that Multi-Head Attention is a powerful tool for social data, provided the head-to-dimension ratio is carefully tuned.