DDQNTrust: Leveraging Reinforcement Learning and Similarity for Robust Trust Inference

TL;DR

The paper proposes DDQNTrust, a framework that uses Double Deep Q-Networks (DoubleDQN) to navigate complex social networks and find reliable trust paths. By combining reinforcement learning with a new aggregation method based on User Interest Similarity, the authors solve the twin problems of excessive memory consumption and low prediction accuracy in local trust inference.

Context & Motivation: The Challenge of Indirect Trust

In modern Online Social Networks (OSNs), users often need to interact with strangers. Trust inference helps determine reliability by propagating trust from a source user to a target user through mutual acquaintances.

Existing methods faces two major hurdles:

1. Scalability: Classical RL methods like Q-learning use "Q-tables," which explode in size as the number of users increases.
2. Accuracy/Path Dependence: Simply averaging trust values along a path ignores whether the intermediate users actually share similar interests or values with the target user.

Methodology: DoubleDQN and SMCFAvg

1. Reliable Path Discovery via DoubleDQN

The authors treat trust propagation as a multi-step decision process.

• State: Each user/node in the network.
• Action: Selecting the next trusted neighbor.
• Reward: Positive reinforcement (+100) for reaching a "neighbor of the target," and penalties (-100) for cycles or dead ends.

To avoid the "overestimation bias" of standard DQN (where the agent might over-rely on a single high-trust edge that doesn't lead to a reliable path), the paper employs DoubleDQN. It uses two networks: one to select the action and another to evaluate its value, ensuring a more stable and realistic trust estimation.

(Note: Refer to the paper's DDQNTrust algorithm flow involving Phase 1: Path Sampling and Phase 2: Path Evaluation via Min-propagation)

2. SMCFAvg: Interest-Aware Aggregation

Once paths are found, the system must aggregate opinions from multiple neighbors. The authors argue that trust is positively correlated with Interest Similarity. They use the Pearson Correlation Coefficient based on movie ratings (in the Filmtrust dataset) to weight the opinions:

$r_{st}=\overline{T_s}+\frac{\sum W_j\cdot sim(u_j,u_t)\cdot (T_{jt}-\overline{T_j})}{Negating\_Factors}$

This formula ensures that if a neighbor has similar tastes to the target user, their opinion carries more weight in the final trust score.

Experimental Validation

The model was tested on the Filmtrust dataset (571 users, 1853 trust relations).

Key Findings:

• Path Reliability: DDQNTrust consistently found paths with higher "strength" than greedy search methods.
• Prediction Performance: By integrating Collaborative Filtering (CF) and interest similarity, the Mean Absolute Error (MAE) was reduced compared to baseline propagation models.

(Note: Experimental results show the advantage of combining RL-based discovery with similarity-weighted aggregation)

Critical Insight & Conclusion

The brilliance of DDQNTrust lies in its hybrid approach. While the DRL component (DoubleDQN) handles the "search problem" of finding paths in a high-dimensional space without storing massive tables, the aggregation component (SMCFAvg) handles the "social logic" of trust.

Limitations:

• The algorithm’s performance is heavily dependent on the availability of rating data to calculate Pearson similarity. In "cold-start" scenarios where ratings are sparse, the performance may degrade.
• Future work could explore using Graph Embedding (like Node2Vec or GCNs) to represent states more richly than simple node IDs.

Takeaway: For researchers in OSNs and recommender systems, this paper proves that moving from "Shortest Path" logic to "Reliable Path" logic via Deep RL is the key to scaling trust systems.