Do AI-generated pull requests improve code quality or only coding speed?

Does AI-generated code actually improve quality, or just speed things up?

The short answer is that AI-generated pull requests (PRs) primarily boost coding speed, but they often degrade code quality and can even slow down the overall review process. The strongest evidence comes from a large-scale analysis of over 33,000 AI-generated PRs, which found that AI agents frequently ignore opportunities to reuse existing code, leading to higher levels of redundancy compared to human developers [8]. This means the code is longer, harder to maintain, and more likely to introduce technical debt—problems that don't show up in simple pass-rate tests but become costly over time.

Another study of 23,247 AI-authored PRs found that 1.7% had serious inconsistencies between the PR description and the actual code changes, with the most common issue being "descriptions claim unimplemented changes" (45.4% of those cases) [1]. These unreliable descriptions directly hurt quality by undermining trust: high-inconsistency PRs had a 51.7% lower acceptance rate (28.3% vs. 80.0%) and took 3.5 times longer to merge (55.8 vs. 16.0 hours) [1]. So while the AI writes code fast, the human reviewer ends up spending more time verifying and fixing it.

A separate investigation of 33,707 agent-authored PRs revealed a stark two-regime pattern: agents excel at narrow, simple automation (28.3% of PRs merge instantly), but frequently fail at iterative refinement, leading to a high rate of "ghosting" (abandonment) when faced with subjective feedback [3]. This creates a hidden "attention tax" on maintainers, who must manage complex interaction loops rather than just reviewing code. The study introduced a model that can predict these high-maintenance PRs before human review begins, capturing 69% of the high-effort PRs with only a 20% review budget [3].

Are AI-generated PRs secure and reliable?

Security is a mixed bag. On one hand, AI-augmented tools like Bugdar can process code reviews in an average of 56.4 seconds per PR (30 lines of code per second), which is dramatically faster than manual reviews that can take hours [4]. This suggests AI can help catch security issues quickly. However, a dedicated study of over 33,000 AI-generated PRs identified 675 security-related submissions and found that they introduce a small set of recurring weaknesses, such as regex inefficiencies, injection flaws, and path traversal [7]. Alarmingly, many flawed contributions were still merged, and rejections often arose from social or process factors like inactivity or missing test coverage, not from the security flaws themselves [7].

The reliability of AI-generated code is also questionable. A study of 21,000 pull requests from 25 popular Java projects found that both accepted (37%) and rejected (44%) PRs contained code smells—suboptimal design choices that often lead to faults [5]. The smelly PRs were more complex, harder to understand, had longer latency times, and were submitted by less experienced contributors [5]. This suggests that AI-generated code may be particularly prone to these quality issues, though the study did not specifically isolate AI-authored PRs.

A promising approach to improving reliability is Test-Driven Prompting (TDP), where example test cases are included in the prompt to the AI. In a study testing 8 different AI models on 3 programming benchmarks, TDP improved code generation success rates by an average of 7.74% across all 16 model-dataset combinations, with a 95% confidence interval of [6.30%, 9.18%] and a very large effect size (Cohen's d = 1.83) [2]. This shows that with the right prompting strategy, AI code quality can be significantly improved, especially for complex or ambiguous tasks.

How do human reviewers actually react to AI-generated PRs?

Surprisingly, human reviewers tend to react more positively to AI-generated contributions than to human ones, even when the AI code is objectively lower quality. An emotion analysis of reviewer comments found that reviewers expressed more neutral or positive emotions towards AI-generated PRs [8]. This disconnect is dangerous: the surface-level plausibility of AI code masks underlying redundancy, leading to the silent accumulation of technical debt in real-world development environments [8].

The review process itself can be streamlined with AI assistance. One study introduced an AI bot that provides initial PR reviews, offering feedback on code formatting, best practices, and minor issues, and automatically assigns reviewers [9]. The goal is to evolve this into an intelligent reviewer capable of assessing code from a functional standpoint, which could significantly enhance PR management and expedite the development workflow [9]. However, the same study notes that this is an ongoing effort, and the bot is not yet fully capable of functional assessment.

A critical finding from the largest study of AI-generated PR rejection patterns is that 84.2% of rejected AI-generated PRs are closed without any inline reviewer feedback—they are simply abandoned [6]. Among those that do receive comments, the dominant visible rejection pattern is functional failures, especially specification mismatch and logic defects [6]. This suggests that many AI-generated PRs are so fundamentally flawed that reviewers don't even bother to explain why, or the PRs are simply ignored. The study also found that documentation co-changes are associated with lower rejection odds, implying that AI agents that include documentation updates are more likely to be accepted [6].

About These Sources

This answer is built on 9 studies (4 peer-reviewed, 5 preprints) — published from 2023 to 2026, 8 from 2024 or later, 2 in Q1–Q2 journals — selected as the most relevant from 10 studies that passed quality screening, drawn from 34 papers retrieved from a database of over 500 million.

Sources used in this answer

Related research questions