When do knowledge graphs actually improve RAG performance?
Knowledge graphs boost RAG performance most when the domain involves complex, multi-step reasoning across connected entities. In a study on Traditional Chinese Medicine diagnosis, a knowledge-graph-grounded RAG system achieved 51.7% recall for syndrome differentiation and 42.5% end-to-end reasoning path completeness—absolute gains of 9.4 and 11.3 percentage points over a zero-shot LLM baseline [2]. The graph captured the clinical reasoning chain (Syndrome → Treatment Principle → Formula), which plain text retrieval could not replicate. Similarly, in a customer service setting, a knowledge graph that preserved intra-issue structure and inter-issue relations improved retrieval accuracy by 77.6% in Mean Reciprocal Rank (MRR) and boosted BLEU text generation scores by 0.32, leading to a 28.6% reduction in median issue resolution time [3]. These gains come from the graph's ability to encode relationships that flat text loses.
The benefit is not universal, however. A study comparing Naïve RAG (plain text retrieval) and Graph RAG across three general-domain benchmarks found that Graph RAG achieved only 8.85% and 15.12% accuracy on two datasets, while Naïve RAG reached 19.04% and 35.32% on the same tasks [1]. The authors noted that Graph RAG handles complex queries well by leveraging relationships, but data must be organized as a knowledge graph for it to reach its potential—otherwise, the graph structure adds noise rather than signal.
Is combining knowledge graphs with text chunks better than using either alone?
Yes, combining both structured knowledge graphs and unstructured text chunks can outperform using either alone, but only if the system adaptively balances them. A study introducing DualRAG found that simply concatenating graph and text information often introduces irrelevant or conflicting content, hurting performance [4]. DualRAG instead adjusts the priority of knowledge from graphs versus chunks based on which contributes more to answering a given query. This adaptive approach significantly improved generation accuracy across multiple benchmarks, suggesting that the best RAG systems don't just add a knowledge graph—they intelligently decide when to rely on it.
Another line of research shows that knowledge graphs can also be used to improve the retrieval step itself, not just the generation. A framework for Knowledge Graph Completion used RAG with LLMs to predict missing triples, achieving superior precision and scalability on benchmark datasets [6]. This bidirectional benefit—graphs improving RAG, and RAG improving graphs—is also highlighted in a comprehensive roadmap that argues for unifying LLMs and knowledge graphs in three ways: KG-enhanced LLMs, LLM-augmented KGs, and synergized systems where both play equal roles [5].
What's the catch—do knowledge graphs make RAG more fragile?
Knowledge graphs can make RAG systems more robust to certain types of noise, but they also introduce new failure modes. A comparative analysis tested GraphRAG against a standard RAG baseline across four robustness scenarios: noise robustness, information integration, negative rejection, and counterfactual robustness [7]. GraphRAG with customizations improved over the baseline, but the study also found that inconsistent retrieved information can negatively affect LLM responses, and the graph structure doesn't automatically fix that. The authors concluded that designing reliable RAG systems for real-world scenarios requires careful handling of retrieval quality, not just adding a graph.
Computational complexity is another practical catch. The study that found Graph RAG underperformed on general benchmarks explicitly noted that 'computing complexity remains a restriction' [1]. Building and querying a knowledge graph requires more upfront effort and processing power than simple text retrieval. This trade-off means that for domains where the data is already well-structured as text (e.g., Wikipedia articles), adding a knowledge graph may not be worth the cost. The roadmap paper [5] also points out that knowledge graphs are difficult to construct and evolve, which challenges existing methods to generate new facts and represent unseen knowledge.
About These Sources
This answer is built on 7 peer-reviewed studies — published from 2024 to 2025, 7 from 2024 or later, 1 in Q1 journals, collectively cited 808 times — selected as the most relevant from 8 studies that passed quality screening, drawn from 51 papers retrieved from a database of over 500 million.
Sources used in this answer
Exploring RAG Solutions to Reduce Hallucinations in LLMs
Graph RAG achieved only 8.85% and 15.12% accuracy on two general-domain benchmarks, underperforming Naïve RAG (19.04% and 35.32%) and a fine-tuned model (72.31% and 88.7%), showing that knowledge graphs don't always help and can hurt when data isn't graph-structured.
Knowledge-Grounded RAG for Diagnosing Spleen-Stomach Disorders in TCM
A knowledge-graph-grounded RAG system for TCM diagnosis achieved 51.7% recall for syndrome differentiation and 42.5% end-to-end reasoning path completeness, outperforming zero-shot LLM baselines by absolute gains of 9.4 and 11.3 percentage points, demonstrating clear benefit in a logic-intensive medical domain.
Retrieval-Augmented Generation with Knowledge Graphs for Customer Service Question Answering
A knowledge-graph-based RAG system for customer service improved retrieval accuracy by 77.6% in MRR and BLEU by 0.32 over a text-only baseline, and after deployment reduced median issue resolution time by 28.6% over six months.
Dual RAG: An Effective Graph-Based RAG Framework with Adaptively Integrating Knowledge Graphs and Chunks
DualRAG, which adaptively prioritizes knowledge from graphs versus text chunks based on query type, significantly improved generation accuracy across multiple benchmarks compared to simple concatenation of both modalities.
Unifying Large Language Models and Knowledge Graphs: A Roadmap
A forward-looking roadmap identifies three frameworks for unifying LLMs and knowledge graphs—KG-enhanced LLMs, LLM-augmented KGs, and synergized systems—and reviews existing efforts, noting that KGs are difficult to construct and evolve.
Enhancing Knowledge Graph Completion with Retrieval-Augmented Generation Using Large Language Models
A RAG-based framework for Knowledge Graph Completion, using LLMs to predict missing triples, achieved superior precision and scalability on benchmark datasets measured by MRR and Hits@10.
Towards Robust Retrieval-Augmented Generation Based on Knowledge Graph: A Comparative Analysis
Customized GraphRAG improved robustness over a standard RAG baseline across four scenarios (noise, integration, negative rejection, counterfactual), but inconsistent retrieved information still negatively affected LLM responses.