How well do LLM pair programmers detect unsafe code?
LLM pair programmers can detect a significant portion of unsafe code, often outperforming traditional static analysis tools. In a study using MUBench (a benchmark of API misuse cases), GitHub Copilot achieved a detection accuracy of 86.2%, with 91.2% precision and 92.4% recall [1]. This means it correctly identified over 9 out of 10 API misuses, and when it flagged something as unsafe, it was right about 91% of the time. For context, the best traditional static code analyzer tested on PHP web vulnerabilities found only 32% of them, while ChatGPT found 62-68% [2]. Even combining multiple traditional tools only reached a 53% detection rate, still well below ChatGPT's performance [2].
Can LLMs fix the unsafe code they find?
When LLMs detect unsafe code, they are often very good at fixing it. In the same Copilot study, once Copilot identified an API misuse, it successfully fixed over 95% of those misuses [1]. This suggests that the model is not just good at spotting problems but also at generating corrected code. However, this high fix rate only applies to the misuses it actually detects — it still misses some, especially complex or context-sensitive cases [1]. Additionally, a separate study found that Copilot's generated code itself often contains vulnerabilities like cross-site scripting and SQL injection [4], meaning the fixes might introduce new problems if not carefully reviewed.
What are the key limitations?
The biggest limitation is that LLMs have very high false positive rates — they flag safe code as unsafe, which wastes developer time. ChatGPT's false positive rate was 91%, meaning that out of every 100 alerts it raised, 91 were incorrect [2]. Even the worst traditional static analyzer had a lower false positive rate (82%) [2]. Another limitation is that LLMs struggle with compound or context-sensitive misuse cases [1], and their generated code often contains serious security flaws [4]. For example, a study of Copilot's code suggestions found that only 57% of Java solutions passed tests, and JavaScript solutions passed only 27% [3]. This means developers cannot blindly trust LLM suggestions and must always review the code.
About These Sources
This answer is built on 5 peer-reviewed studies — published from 2022 to 2025, 3 from 2024 or later, collectively cited 269 times — selected as the most relevant from 5 studies that passed quality screening, drawn from 52 papers retrieved from a database of over 500 million.
Sources used in this answer
Can We Trust the AI Pair Programmer? Copilot for API Misuse Detection and Correction
GitHub Copilot detected API misuses with 86.2% accuracy, 91.2% precision, and 92.4% recall on the MUBench benchmark, and fixed over 95% of the misuses it identified.
New Tricks to Old Codes: Can AI Chatbots Replace Static Code Analysis Tools?
ChatGPT found 62-68% of PHP web vulnerabilities, outperforming the best traditional static analyzer (32%) and even combined tools (53%), but had a very high false positive rate of 91%.
An empirical evaluation of GitHub copilot's code suggestions
In an empirical evaluation using 33 LeetCode questions across 4 languages, Copilot's Java suggestions had the highest correctness (57%) and JavaScript the lowest (27%), with some code relying on undefined helper methods.
CodingCare: AI Code Generation Security Framework for Common Vulnerability Mitigation
A review found that AI-generated code contains serious vulnerabilities like cross-site scripting and SQL injection, and proposed a security framework that reduced reported CVEs.
Demo: Fixing C/C++ Vulnerabilities with LLMs : Prompt, Detect, Fix: No More Unsafe Code
Locally hosted LLMs (e.g., Deep Seek) can scan entire codebases for vulnerabilities and propose fixes, reducing risks of backdoors associated with remote LLMs like ChatGPT.