[KDD 2025] GuARD: Bridging the Gap Between Graph Topology and LLM Semantics for Anomaly Detection

Executive Summary

TL;DR: GuARD (Graph-informed and text-Rich language model for Anomaly Detection) is a novel framework that enhances LLMs with the ability to "see" graph structures and "digest" massive textual attributes. By using a progressive multi-turn instruction tuning strategy, it achieves SOTA accuracy while delivering up to 10x faster inference than traditional LLM fine-tuning.

Background: In the landscape of anomaly detection, we have long been split between two worlds: GNNs, which excel at structural patterns but miss semantic nuances, and LLMs, which understand text deeply but are blinded by graph topology and crushed by long-context overhead. GuARD serves as a sophisticated bridge between these two paradigms.

Problem & Motivation: The "Long-Context" Trap

Detecting anomalies in text-rich graphs—such as identifying incorrectly assigned papers in academic networks or spotting bots on social media—requires analyzing both node relationships and dense attributes (titles, abstracts, metadata).

Current SOTA methods face a catch-22:

1. GNNs (e.g., GCCAD): Often simplify text into shallow embeddings, losing the "reasoning" capability of modern LLMs.
2. LLMs (e.g., Llama 3): When fed 10k+ tokens of node data, they suffer from the "lost in the middle" phenomenon and become computationally unsustainable for real-time production.

The authors' key insight is that an LLM doesn't need to read every word of a node's metadata if those attributes can be summarized into specialized "semantic tokens" and "structural tokens."

Methodology: The Three Pillars of GuARD

GuARD doesn't just jam data into a prompt. It employs a Progressive Multi-modal Instruction Tuning framework:

1. Multi-Turn Task Alignment

Instead of predicting one node at a time, GuARD uses a multi-turn chat template. This allows the model to reuse global context (like an author's profile) across multiple paper queries, significantly boosting inference speed through KV-cache reuse.

2. Semantic Embedding Module (The "Text" Token)

A smaller, efficient encoder (like RoBERTa or DeBERTa) processes the "heavy" text. This summary is projected through a gated FFN into a special <text> token that the LLM consumes.

3. Structural Embedding Module (The "Graph" Token)

To fix the "structural blindness" of LLMs, GuARD extracts topological features using a GNN (e.g., HGT) and injects them as a <graph> token. This gives the LLM a direct signal about where the node sits in the network.

Experiments: Speed Meets Precision

GuARD was tested against heavyweights like Llama 3-8B and Qwen 2.5-7B across datasets including WhoIsWho and TwiBot-20.

• Accuracy Boost: On WhoIsWho, GuARD-base (75.7 AUC) jumped to 78.9 AUC after adding semantic and structural modules.
• Efficiency Gains: The model achieved a 10x speedup in inference compared to vanilla LLM approaches. By summarizing context into tokens, the memory footprint is drastically reduced.

The Progressive Strategy

A critical finding of the paper is the Progressive Training (tuning the base, then semantic, then graph modules sequentially). The authors found that training everything at once (from-scratch) led to the model ignoring semantic signals in favor of easier-to-learn graph features.

Critical Analysis & Conclusion

GuARD represents a pragmatic shift in AI architecture. Instead of building "wider" windows (Long Context), it builds "deeper" representations (Multi-modal Summary Tokens).

Takeaways:

• Inductive Bias Matters: LLMs are great at zero-shot reasoning but need external help to understand graph topology.
• Efficiency is King: Multi-turn instruction tuning is a powerful tool for context sharing that should be standard in GNN-LLM hybrids.

Limitations: The framework still relies on a "frozen" GNN and SLM. A fully end-to-end differentiable pipeline across all three scales (GNN $\to$ SLM $\to$ LLM) might unlock even higher performance, though at a significant training cost.

For researchers in bot detection, fraud analysis, or academic disambiguation, GuARD provides a robust, scalable framework that respects both the word and the link.