Which is better for teaching a model new facts?
If your goal is to get a model to answer questions about facts it wasn't trained on, retrieval-augmented generation (RAG) is the clear winner. RAG works by pulling relevant documents from an external source and feeding them to the model at answer time, so the model doesn't have to memorize the fact. In a 2024 study comparing the two approaches across many knowledge-intensive tasks, RAG consistently outperformed unsupervised fine-tuning, both for facts the model already knew and for entirely new information [5]. Another study focusing on less-popular knowledge found that RAG beat fine-tuning by a large margin, especially for the least popular facts [4].
Fine-tuning, by contrast, struggles to teach models new factual information. The same 2024 study found that models often fail to learn new facts through unsupervised fine-tuning, and only improved when they were shown many variations of the same fact during training [5]. This is a crucial limitation: if you need to update a model with a specific new fact, RAG is more reliable and cheaper than trying to fine-tune it in.
When does fine-tuning still make sense?
Fine-tuning isn't useless—it's just better at changing how a model behaves rather than what it knows. For example, fine-tuning can turn a general-purpose language model into a specialized retriever or reranker. In a 2024 study, researchers fine-tuned LLaMA-2 to act as a dense retriever and a pointwise reranker, and the resulting pipeline outperformed smaller models on passage and document retrieval tasks [3]. This shows that fine-tuning can optimize the model's internal representations for a specific task, not just add facts.
Fine-tuning also helps smaller models catch up. The same study on less-popular knowledge found that fine-tuning was beneficial for small language models, even though it required extensive resources [4]. And in the medical domain, fine-tuning alone was sometimes competitive, but the best results came from combining fine-tuning with RAG [2]. So if you have a small model and need it to perform a specific task, fine-tuning is worth considering—but for adding new knowledge, RAG is the safer bet.
Is it better to combine RAG and fine-tuning?
Often, yes. The medical study that compared RAG, fine-tuning, and both together found that RAG and the combination consistently outperformed fine-tuning alone across most models, with the combination often being the strongest [2]. This suggests that fine-tuning can help the model use retrieved information more effectively, while RAG supplies the up-to-date knowledge.
However, combining both is more complex and resource-intensive. The same study noted that fine-tuning requires extensive resources, and the authors proposed a new 'Stimulus RAG' approach that matched or beat fine-tuning without the need for costly fine-tuning steps [4]. So while combining can yield the best performance, it's not always necessary—RAG alone may be enough for many use cases, especially when you're working with a large, capable model.
About These Sources
This answer is built on 5 peer-reviewed studies — published from 2024 to 2025, 5 from 2024 or later, 2 in Q1–Q2 journals, collectively cited 207 times — selected as the most relevant from 6 studies that passed quality screening, drawn from 34 papers retrieved from a database of over 500 million.
Sources used in this answer
Fine-Tuning Retrieval-Augmented Generation with an Auto-Regressive Language Model for Sentiment Analysis in Financial Reviews
Developed LFEAR, a model combining RAG with auto-regressive fine-tuning, achieving 98.45% precision, 93.85% answer correctness, and 97.69% context precision on sentiment analysis across domains.
Medical LLMs: Fine-Tuning vs. Retrieval-Augmented Generation
In a comparison on medical QA, RAG and RAG+fine-tuning consistently outperformed fine-tuning alone across most of five models, with LLAMA and PHI showing the strongest results.
Fine-Tuning LLaMA for Multi-Stage Text Retrieval
Fine-tuned LLaMA-2 as a dense retriever and reranker, showing that fine-tuned LLM retrieval models outperform smaller models and can handle long documents without heuristic segmentation.
Fine Tuning vs. Retrieval Augmented Generation for Less Popular Knowledge
Across 12 language models, RAG surpassed fine-tuning by a large margin for less-popular factual knowledge, while fine-tuning helped small models but required extensive resources; proposed Stimulus RAG to match fine-tuning without the cost.
Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMs
Comparing unsupervised fine-tuning and RAG on knowledge-intensive tasks, found RAG consistently outperformed fine-tuning for both existing and new knowledge, and that models struggle to learn new facts via fine-tuning unless shown many variations.