IFMBert: Tracking User Preference through Social Influence and Bidirectional Sequences

TL;DR

Recommending the next "Point of Interest" (POI) is a cornerstone of Location-Based Social Networks (LBSN). This paper introduces IFMBert, a hybrid model that marries the feature-interaction strengths of Factorization Machines with the sequence-understanding power of BERT. By quantifying social trust and modeling check-in history bidirectionally, IFMBert tackles the twin demons of recommendation: data sparsity and interest drift.

Problem & Motivation: Beyond Simple Check-ins

Why is POI recommendation harder than recommending a movie or a book?

1. Heterogeneity: You aren't just dealing with user IDs; you have timestamps, GPS coordinates, social circles, and category metadata.
2. Sparsity: Most users only visit a tiny fraction of available POIs.
3. Interest Drift: Your interest in a "Breakfast Spot" is highly dependent on your previous night’s "Hotel" check-in and your social circle's expert opinions.

Current SOTA methods like GRU4Rec are unidirectional (left-to-right), missing out on the full context of a user's behavior pattern. Meanwhile, standard Factorization Machines (FM) treat social links as binary (0 or 1), failing to recognize that some friends are more "expert" than others.

Methodology: The Fusion of Social Trust and BERT

The authors propose a two-pronged architecture to capture the "Who" (Social) and the "When/Where" (Sequential).

1. Improved Factorization Machine (IFM)

Instead of a vanilla FM, the authors inject Social Influence ($S_{u,v}$). This is calculated using two metrics:

• Social Closeness: Based on graph theory (in-degree and out-degree), determining how much weight User A's opinion should carry.
• User Expertise: A specific metric calculating how knowledgeable a friend is in a specific POI category (e.g., if a friend has checked into 50 Ramen shops, their Ramen recommendation carries more weight).

2. Sequential Modeling with BERT

The model treats a sequence of check-ins like a "sentence." Using a bidirectional Transformer, the model looks at the entire check-in history at once. To prevent "cheating" during training (information leakage), they use the Cloze Task (Masked LM) approach, masking items and forcing the model to predict them based on surrounding context.

Figure 1: Overview of the IFMBert architecture merging sequence features and social FM features.

Experiments & Results

The model was validated on the Yelp dataset (covering NC and WI regions), which provides a rich tapestry of ratings and social links.

Performance Comparison

IFMBert consistently outperformed traditional baselines:

• Against FM/DeepFM: The addition of BERT's sequential features allowed it to understand the order of visits, not just the features of the visit.
• Against GRU4Rec: The bidirectional nature of BERT provided a more holistic view of user intent than the unidirectional RNN approach.

Table 1: IFMBert shows a significant lead in HR@10 and NDCG@10.

Ablation Insights

The ablation study revealed that:

1. Removing Social Influence leads to a drop in performance, especially for "cold-start" scenarios where the check-in history is thin.
2. Removing BERT (Sequence) makes the model "blind" to the evolution of user interest, proving that the order of check-ins is vital for predicting the next step.

Critical Analysis & Conclusion

Takeaway

The core insight of this paper is that Social Influence is not binary. By quantifying expertise and closeness, we can "densify" sparse check-in data. Furthermore, treating LBSN data as a language—where check-ins are words and sequences are sentences—allows the recommendation field to inherit the massive gains made in NLP via BERT.

Limitations & Future Work

While IFMBert is powerful, it primarily treats location as a discrete token. Future iterations could benefit from Spatial-Temporal embeddings (explicitly modeling the distance between POIs and the time elapsed between check-ins) to further refine the "geographical" logic of the recommendations.

Keywords: POI Recommendation, LBSN, Factorization Machines, BERT, Sequence Modeling.