Do AI-generated pull requests change how software engineering should be taught?

What do AI pull requests reveal about the new skills students need?

AI agents don't write code the way humans do, and that changes what students need to learn. A study of 1,254 performance-related code changes from AI agents found that 37% were name modifications, 26.4% were object creation, and 22.7% were type changes—a pattern completely different from human-written code, where 84% of changes are "no change" (meaning humans mostly tweak existing code) [1]. This means students can't rely on intuition about what a "typical" code change looks like; they need to learn how AI agents think and what kinds of mutations they produce.

Merge conflicts are another new reality. A dataset of 142,000+ AI-generated pull requests found that 27.67% of them caused merge conflicts—meaning nearly 3 out of every 10 AI contributions can't be cleanly integrated [7]. That's a much higher rate than most human PRs, and it varies by agent type. Students now need explicit training in conflict resolution and integration testing, not just how to write code that compiles.

AI agents also fail in predictable ways that students should learn to spot. An analysis of 12,433 AI-generated PRs found that 84.2% of rejected PRs were closed without any reviewer feedback—silent failures [2]. Among those that did get comments, the top reasons were specification mismatch and logic defects. This means students need to develop the ability to detect when an AI has misunderstood the requirements or introduced subtle logical errors, skills that aren't typically taught in standard programming courses.

How should software engineering courses adapt to AI-generated contributions?

The evidence points to a shift from "write code" to "manage AI contributions." A 2022 survey of 248 studies on software engineering for AI-based systems found that testing and quality are heavily researched, but software maintenance is neglected [5]. This gap is critical because AI agents generate code that needs constant maintenance and review. Courses should integrate AI-assisted pull request review tools, as a 2025 study design proposes, to teach students how to evaluate AI-generated changes against requirements [3].

Context-aware AI feedback is more effective than generic AI tutoring. A controlled study with 26 students found that when AI review tools were given project context (like PR descriptions and comments), students rated the feedback as more accurate and useful [4]. This suggests that teaching students how to provide good context to AI tools—and how to interpret context-aware feedback—is a learnable skill that improves outcomes.

However, AI tutoring has limits. A case study integrating GPT-3.5 as an AI tutor in a programming assessment system found that while students appreciated timely feedback and scalability, they also reported concerns that the AI could inhibit their learning progress by giving generic responses [8]. This means educators can't just hand students an AI tool and expect it to teach; they need to design assignments where students actively critique and improve AI-generated code, not just accept it.

Where do the studies disagree, and what does that mean for teaching?

The studies agree that AI changes the landscape, but they disagree on whether the change is mostly positive or mostly a new source of problems. On one hand, a 2026 study found that AI-generated PR rejection odds have declined significantly over time, suggesting agents are getting better at meeting repository standards [2]. On the other hand, the same study found that 84.2% of rejections are silent—no feedback given—which means students can't learn from failures if they don't know why the PR was rejected.

Another tension: AI agents optimize performance mostly during development, not maintenance [6]. This is good for rapid prototyping but bad for long-lived software. A survey of 248 studies confirms that software maintenance for AI-based systems is neglected in research [5]. So if educators only teach students to use AI for initial development, they'll miss the harder skill of maintaining AI-generated code over time.

The conflict isn't really a contradiction—it's a reflection of different phases of AI adoption. Early studies (2022-2024) emphasize risks and failures [5][8], while later studies (2025-2026) show improving acceptance rates but persistent integration challenges [2][7]. For teaching, this means curricula should be updated now to cover both the current failure modes and the emerging best practices, because the field is moving fast.

About These Sources

This answer is built on 8 peer-reviewed studies — published from 2022 to 2026, 7 from 2024 or later, 1 in Q1 journals, collectively cited 101 times — selected as the most relevant from 8 studies that passed quality screening, drawn from 56 papers retrieved from a database of over 500 million.

Sources used in this answer

Related research questions