Most studies are too short to prove AI tutoring really works
The biggest gap is simple: we don't have enough long-term evidence. A systematic review of 20 K-12 studies found that half of them lasted less than a single class period — what the researchers called 'very short' interventions [3]. That's like judging a new teaching method after 45 minutes. Even the studies that ran longer often had small groups of students, making it hard to trust the results. Across all 20 studies, only 2,853 students were included total, spread across different subjects and grade levels [3]. This means we have almost no data on whether AI tutoring helps students retain knowledge weeks or months later.
The problem is worse in specialized fields. A meta-analysis of AI tutoring for surgical skills found only 4 studies with 268 participants that met rigorous standards for analysis [1]. The authors rated the evidence as 'low certainty' — meaning the apparent small advantage for AI (0.20 points on a skills test) could easily vanish with better research. When the evidence base is this thin, it's impossible to say whether AI tutoring is genuinely effective or just looks good in short, controlled settings.
We can't measure whether AI tutors actually build deep understanding
A second critical gap is that current tests don't capture what we really care about: deep conceptual understanding and the ability to think about one's own thinking (metacognition). A literature review on AI tutors for high school math found that researchers have been slow to develop good ways to measure these deeper skills [2]. Instead, most studies rely on test scores or task completion times, which can improve even when students don't truly understand the material. The review notes that math scores on Japan's national exam fluctuate by ±20 points year to year — three to five times more than other subjects — suggesting that surface-level tutoring may not address the real learning challenges [2].
This measurement gap matters because AI tutors are often designed to give immediate feedback and adapt to student responses. But if we only measure whether a student got the right answer, we might miss that the AI is simply teaching test-taking tricks rather than genuine understanding. A separate review of feedback in intelligent tutoring systems identified this as a primary gap: the lack of research on whether AI feedback actually improves long-term learning or just short-term performance [5].
We don't know how teachers and AI should work together
Perhaps the most practical gap is that almost no research has tested how human teachers and AI tutors should collaborate. The surgical skills meta-analysis found that AI tutoring imposed a significantly higher extraneous cognitive load on learners — meaning students had to work harder mentally to filter out irrelevant information [1]. The authors concluded that the evidence 'does not support replacing human instructors with AI' and instead points toward a hybrid model, but they immediately note that 'this itself requires rigorous empirical validation' [1]. In other words, the most promising approach — combining human and AI — has barely been studied.
The math education review echoes this, listing 'lack of demonstration of cooperative operation models between teachers and AI' as one of its four main research gaps [2]. This is a real-world problem: schools considering AI tutors need to know whether the AI should act as a teaching assistant, a homework helper, or a full replacement for certain lessons. Without studies that directly compare different collaboration models, schools are left guessing. The K-12 review also notes that ethical implications of using AI for teaching 'should be investigated' [3], another area where evidence is essentially absent.
About These Sources
This answer is built on 5 peer-reviewed studies — published from 2024 to 2026, 5 from 2024 or later, 1 in Q1 journals — selected as the most relevant from 5 studies that passed quality screening, drawn from 61 papers retrieved from a database of over 500 million.
Sources used in this answer
Artificial intelligence augmented tutoring vs expert instruction on learning simulated general surgical skills: a systematic review and meta-analysis.
In a meta-analysis of 4 studies (268 participants), AI tutoring showed a small but statistically significant improvement in surgical skills (0.20 points on a 5-point scale) but also increased extraneous cognitive load; the evidence was rated low certainty and the authors concluded that hybrid human-AI models need rigorous testing.
A Study on Individualized Learning Support for High School Mathematics by an Interactive AI Tutor : Educational Potential and Challenges in the Age of Large Language Models
A literature review on AI tutors for high school math identified four research gaps: lack of validation for struggling students, no LLM-based tutor studies in Japanese high schools, poor measures of deep understanding and metacognition, and no tested models for teacher-AI collaboration.
Navigating the Future of Learning: A Systematic Review of AI-Driven Intelligent Tutoring Systems (ITS) in K-12 Education
A systematic review of 20 K-12 studies (2,853 students) found that half used very short interventions (less than one class period), and while AI tutors generally showed positive effects versus traditional teaching, those effects shrank when compared to non-intelligent computer tutors.
A Comprehensive Review of Intelligent AI Tutoring Systems with Personalized Content Recommendation Using Hybrid ML Models.
A review of hybrid machine learning models for personalized content recommendation in AI tutoring identified key gaps including cold-start problems and the need for continuous learner profiling, proposing a framework that combines collaborative and content-based filtering.
Use of feedback in intelligent tutoring systems: a systematic literature review
A systematic literature review on feedback in intelligent tutoring systems identified primary research gaps including the lack of studies on whether AI feedback improves long-term learning versus just short-term performance.