Mining Graphlet Counts: Efficient Sampling in Restricted Online Social Networks

TL;DR

Analyzing the "DNA" of a social network—its small induced subgraphs or graphlets—is crucial for understanding social behavior, but doing so on massive graphs like Twitter or Weibo is a computational nightmare. This paper introduces a highly efficient, random walk-based framework that estimates 3-, 4-, and 5-node graphlet counts with high precision using only a tiny fraction of the network data. It bridges the gap between theoretical MCMC methods and the practical constraints of OSN APIs.

Problem & Motivation

Why is counting subgraphs so hard?

1. Combinatorial Explosion: The number of potential $k$-node subgraphs grows exponentially. For a moderate Twitter graph, exact counting of 4-node graphlets can take over a week.
2. Restricted Access: Researchers usually don't have the raw database. They must "crawl" the graph via APIs, which are rate-limited and expensive.

Previous works often focused on relative frequencies. However, knowing the absolute count is vital for anomaly detection and graph comparison. The authors' intuition was to use the "visible" information around a random walker more effectively—not just the nodes visited, but their entire local neighborhood observed during the walk.

Methodology: The Core

The framework moves beyond simple node sampling. It treats a sequence of $k-1$ consecutive steps in a random walk as a "touched" subgraph, which then "observes" $k$-node subgraphs in its immediate vicinity.

1. The Expanded Markov Chain

The authors map the random walk to an Expanded Markov Chain. If we want 4-node graphlets ($k=4$), the state space ${\mathcal{M}}^{(3)}$ consists of all possible 3-step paths. This allows the use of the Strong Law of Large Numbers (SLLN) to ensure that the average of sampled observations converges to the true population mean.

2. Importance Sampling & Re-weighting

Not all subgraphs are equally likely to be seen by a random walker. High-degree nodes (hubs) are visited more often. To fix this, the authors derive a re-weighting function $w_i^k(X)$ based on the stationary distribution ${\pi }_M$ of the expanded chain and the number of ways a specific graphlet can be "found" (${\beta }_i^k$).

Figure: The General Access Model (a) vs. the Special Access Model (b) which provides degree information of neighbors.

3. Improved Estimators (ImprG & ImprS)

The real "secret sauce" lies in the improved estimators. By treating all paths that cover the same set of nodes as a single subgraph unit, they reduce the variance caused by node ordering, leading to a more stable estimate—essentially getting more "signal" out of the same number of API calls.

Experiments & Results

The authors tested their method against massive datasets, including Twitter (21M nodes) and Weibo (58M nodes).

• Accuracy: With just 20,000 nodes (a minuscule fraction of Weibo), the error for triangle counts is under 5%.
• SOTA Comparison: Against previous models like PSRW, the proposed ImprG estimator is significantly more accurate, especially for complex structures like 4-node cliques ($g_6^4$).

Figure: Convergence of the estimator. As sample size (n) increases, the estimates (LB/UB) rapidly tighten around the ground truth (1.0).

Key Ablation: Known vs. Unknown Edge Counts

In real-world OSNs, you rarely know the total number of edges $|E|$. The authors proved that their method works even when $|E|$ is estimated during the walk, with negligible loss in accuracy.

Analytical Insight & Conclusion

The beauty of this work is its mathematical rigor. The authors provide an analytical bound on the sample size required for a desired accuracy $(1\pm ϵ)$, factoring in the mixing time of the graph. For "small-world" social networks where mixing is fast, this explains why the algorithm converges so quickly.

Takeaway: This paper provides a blueprint for large-scale social data mining. By combining Markov chain theory with clever local neighborhood observations, it proves that we can "see" the global structural properties of a massive network by only looking at a tiny, strategically sampled corner of it.