T-ABT: Efficient Queryable Compression for Massive Streaming Social Networks

TL;DR

Managing time-evolving graphs—where relationships flash in and out of existence—is a memory nightmare. This paper presents T-ABT, a compression scheme that treats temporal graphs as 3D matrices. It achieves a 77% reduction in storage (e.g., 21.5GB to 4.9GB) while remaining "queryable" (read without decompressing) and "streaming" (update without decompressing).

Background & Motivation

In the era of billion-node social networks, even a single snapshot can overwhelm RAM. When you add the temporal dimension (who was friends with whom at 2:00 PM last Tuesday?), the data complexity explodes.

Previous state-of-the-art (SOTA) methods like ckd-tree or k2-tree variants often struggle with two things:

1. Intermediate Bloat: They need to build a massive adjacency list before they can compress it.
2. Inactivity Bursts: Standard trees are great at pruning "zeros" (no connection) but poor at pruning "ones" (long-lasting connections).

The authors' insight was to move away from static compression toward an alternating logic that adapts to both presence and absence of activity over time.

Methodology: The Alternating Binary Tree

The core innovation is the Alternating Compressed Binary Tree.

1. 2D-to-3D Projection

The algorithm first flattens the time-evolving graph into an aggregated 2D matrix. Each "cell" in this matrix isn't just a bit; it's a pointer to a nested binary tree representing that specific edge's lifetime.

2. The Alternating Strategy

Standard tree compression prunes branches that contain only zeros. T-ABT goes further:

• If a time range contains only zeros, the branch is pruned.
• If a time range contains only ones, the tree "flips" its logic. Instead of looking for ones, it now looks for the next zero.
• This allows the structure to compress long "runs" of activity just as efficiently as runs of inactivity.

Fig 1: The root node determines if it represents edges (1s). If a leaf is reached, the logic flips to represent 0s, adapting to data runs.

3. "Zero-Memory" Compression

To solve the mountain-of-data problem, the authors developed a way to compress directly from a gzipped text file. Using the zLib library, they decompress only one node’s worth of data at a time, compress it into T-ABT, and move on. This prevents the "21.5GB problem" from ever hitting the RAM.

Experiments & Results

The authors tested T-ABT on massive datasets including Wikipedia links and Yahoo! network flows.

Fig 2: Overall structure of the T-ABT aggregation and nested compression.

Performance Highlights:

• Query Speed: Edge queries (checking if node A and B are connected at time T) take sub-millisecond times, even on the 100-million-node Yahoo dataset.
• Streaming: Adding or removing an edge (streaming) is nearly instantaneous ($<0.1$ms), making it viable for live system monitoring.

Critical Insight & Conclusion

T-ABT’s power lies in its structural simplicity. By nesting binary trees, it mimics the "natural" sparsity of social networks—where most people aren't connected, but those who are often stay connected for long periods.

Limitations: The performance depends heavily on the "lifetime" of the graph. As the temporal resolution increases (e.g., measuring by the millisecond over years), the tree depth grows, which can slow down queries.

Future Outlook: The authors suggest "partitioning the lifetime"—breaking a long-running graph into chunks and differentially encoding them. This could potentially allow T-ABT to scale to decades of social platform data without losing its edge.