How much can AI help without flooding the team?
The evidence shows that AI security PRs can be both effective and low-friction when designed for a narrow, well-defined task. Dependabot, which automatically updates vulnerable dependencies, is a clear success story: across 2,904 JavaScript projects, 65.42% of its security PRs were accepted, and most were merged within a day [4]. Only 3.2% of those PRs caused build breakages, meaning the vast majority were safe to merge without extra work [4]. This suggests that for routine dependency updates, AI can handle security fixes with minimal team burden.
For broader code-review tasks, newer AI tools also show promise. Bugdar, an AI-augmented code review system for GitHub, processes an average of 56.4 seconds per PR (about 30 lines of code per second), which is dramatically faster than manual reviews that can take hours [1]. It uses fine-tuned large language models and retrieval-augmented generation to give project-specific feedback, which should reduce irrelevant alerts [1]. However, this tool is very new (2025) and hasn't been independently validated at scale yet, so its real-world false-positive rate is still unproven.
What are the caveats and risks?
Not all AI security PRs are created equal. A large-scale study of over 33,000 AI-generated PRs found 675 that were security-related, and these introduced recurring weaknesses like regex inefficiencies, injection flaws, and path traversal [3]. Worryingly, many flawed contributions were still merged, and rejections often came from social or process factors (like inactivity or missing tests) rather than security flaws [3]. This means AI can introduce new vulnerabilities even while trying to fix others, and teams may not catch them.
The same study found that commit message quality—often a sign of careful work—had little effect on whether an AI PR was accepted or how fast it was merged, unlike with human PRs [3]. This suggests that teams may not be scrutinizing AI contributions as carefully as they should. The authors also extended existing rejection taxonomies with categories unique to AI-generated security PRs, indicating that AI introduces novel failure modes that teams need to learn to recognize [3].
Who benefits most, and under what conditions?
Teams that benefit most are those using AI for narrow, repetitive security tasks like dependency updates. Dependabot's high acceptance rate and low breakage rate show that this is a low-risk, high-reward use case [4]. Projects with prior experience using Dependabot also saw faster merges, meaning teams that invest in learning the tool get compounding benefits [4].
For broader security code review, the conditions for success are stricter. Bugdar's design—using project-specific tuning and supporting multiple languages—suggests that customization is key to avoiding false positives [1]. Teams should expect to invest time in configuring the AI to their codebase and reviewing its output, at least initially. The study of 33,000 AI PRs also found that security-related AI PRs tend to have a small set of recurring weaknesses, so teams can create targeted checklists or additional automated checks to catch those specific issues [3].
A critical condition is that teams must not blindly trust AI. The fact that flawed security PRs get merged [3] means human review remains essential, especially for non-routine changes. The ideal setup is a partnership: AI handles the grunt work (dependency updates, routine pattern checks) while humans focus on architecture-level security decisions and novel vulnerabilities.
About These Sources
This answer is built on 4 studies (2 peer-reviewed, 2 preprints) — published from 2021 to 2026, 3 from 2024 or later — selected as the most relevant from 4 studies that passed quality screening, drawn from 41 papers retrieved from a database of over 500 million.
Sources used in this answer
Bugdar: AI-Augmented Secure Code Review for GitHub Pull Requests
Bugdar, an AI-augmented code review system for GitHub PRs, processes a PR in an average of 56.4 seconds (30 LOC/second), uses fine-tuned LLMs and RAG for project-specific feedback, and supports Solidity, Move, Rust, and Python. It aims to reduce false positives by tailoring analysis to each codebase.
Replication Package: "Insights into Security-Related AI-Generated Pull Requests"
This is a replication package (datasets, scripts, documentation) for an empirical study of security-related AI-generated PRs. It includes manual annotation of 245 PRs and inter-annotator agreement analysis, but the abstract provides no quantitative findings.
Insights into Security-Related AI-Generated Pull Requests
Analyzed over 33,000 AI-generated PRs, identifying 675 security-related ones. Found recurring weaknesses (regex inefficiencies, injection flaws, path traversal), that many flawed PRs are still merged, and that commit message quality has little effect on acceptance or latency for AI PRs. Extended rejection taxonomies with AI-specific categories.
On the Use of Dependabot Security Pull Requests
Studied Dependabot across 2,904 JavaScript projects: 65.42% of security PRs were accepted, most merged within a day, and only 3.2% caused build breakages. Prior experience with Dependabot was associated with faster merges; vulnerability severity was not strongly associated with merge time.