MultiView: Scaling Standing Queries on Social Networks with 500 Million Edges

TL;DR

As social media generates millions of updates daily, monitoring complex behavioral patterns (e.g., fraud detection or marketing trends) requires high-performance "standing queries." This paper presents MultiView, a system that merges hundreds of overlapping subgraph queries into a unified structure, slashing maintenance costs by 477% and outperforming traditional RDBMS by orders of magnitude.

Background: The Price of Independence

In a massive social network, users might register hundreds of different "views"—for example, detecting illegal file sharing or tracking expert mentions. Currently, systems treat these views as isolated entities. If 100 queries all check for the same "User -> Follows -> Topic" pattern as a prerequisite, the system calculates that specific join 100 times. In a "Huge Social Network" context (like the 540M-edge Orkut dataset used here), this redundancy is the primary bottleneck for real-time performance.

Methodology: Merging the Forest into a Tree

The authors' core "Insight" is that view maintenance should start at the point of change.

1. Merged View Construction

Instead of maintaining a single global graph of queries, the authors create Edge-Annotated Views. When a new edge with label l is inserted, the system only activates the merged graph centered on label l.

Figure: The mapping of two query graphs into a common substructure.

2. The AddView Algorithm & Scoring

Since finding an "Optimal Merge" is NP-hard (reducible from Subgraph Isomorphism), the authors developed AddView. It uses a greedy approach to align new views to existing ones.

• Scoring Logic: It squares the overlap count—rewarding edges that satisfy the highest number of queries simultaneously.
• Efficiency: The "Top-k" greedy version of AddView achieves ~99% of the optimal score while being 1,000x faster than the exact search.

3. MultiView Maintenance

Once merged, the MultiView algorithm uses a depth-first search to traverse the merged graph. By solving the common subgraphs first, it "multicasts" results to all relevant views, avoiding the redundant path-finding typical of the baseline ViewBasic approach.

Figure: Comparing the time complexity and quality of greedy vs. optimal merging.

Experimental Battlefront: Real-World Social Media

The authors tested their system against six diverse datasets, ranging from sparse email networks to dense media networks (YouTube, Flickr).

• Scalability: On the Orkut dataset (540M edges), MultiView handled 80 concurrent views with ease, while RDBMS-based solutions crashed after running out of temp disk space (90GB limit).
• Gain Factor: The performance gain increases linearly with the number of views. At 80 views, the "overlap benefit" makes MultiView nearly 5x more efficient than any single-query optimizer.

Table: The diverse real-world datasets used to validate the MultiView framework.

Critical Insight & Limitations

The primary contribution is the shift from per-query optimization to workload-level optimization. By recognizing that standing queries in a social network aren't random but often target similar "motifs," the authors converted a scaling problem into a compression problem.

Limitations:

1. Insertions-Only: The current algorithm focuses on edge additions. In the real world, "unfollowing" or "unfriending" (deletions) is common and would require a different indexing strategy to handle "decrementing" view counts.
2. Memory constraints: Significant RAM is needed for high-complexity merged graphs, though the authors demonstrated success with as little as 4GB on large clusters.

Future Outlook

This work sets the stage for Multiple Query Optimization (MQO) in graph databases. As GNNs and real-time graph stream processing become standard in industry, the concepts of "Center Edges" and "Merged View Scoring" will likely influence how we design next-generation real-time graph engines.