Are the privacy and fairness risks of AI tutoring systems being underestimated?

How worried are students themselves about privacy and fairness?

Students are not naive about the risks. A 2025 survey-based study found that privacy concerns were a significant predictor of students' overall ethical apprehension about AI in education — meaning the more students worried about how their data was collected and shared, the more ethically uneasy they felt about the whole system [4]. Interestingly, the same study found that students with more prior experience using AI tools actually had more nuanced ethical concerns, not fewer. This suggests that familiarity doesn't breed complacency; it breeds sharper awareness of what could go wrong. The study also showed that fairness, transparency, and data privacy practices were all top-of-mind for students, indicating that the risks are not abstract or distant to the people actually using these systems.

What specific new risks are emerging that might be overlooked?

Two papers highlight risks that go well beyond standard data privacy concerns. First, deepfake-style AI tutors — which use synthetic video or voice to deliver personalized instruction — introduce risks of impersonation, assessment fraud, and algorithmic bias. A 2025 systematic review of 42 peer-reviewed studies found that current detection methods (which rely on pixel-level artifacts, frequency features, or physiological signals) remain imperfect, meaning a bad actor could plausibly impersonate a tutor or a student [2]. Second, for minors aged 13–17 using AI tutors inside game-based learning environments, the risks are amplified by the very features that make these tools engaging: extended play sessions, emotional investment in virtual creations, and parasocial relationships with AI companions. A 2026 safety framework paper argues that these immersive, informal contexts create psychological and privacy vulnerabilities that traditional classroom-focused AI guidelines simply don't address [3]. The paper notes that existing frameworks from regulators (like COPPA 2025 updates, GDPR Article 8, and the EU AI Act) and organizations like UNICEF provide a starting point, but they were not designed for the specific dynamics of game-based tutoring.

Are current governance and fairness protections adequate?

No — the evidence points to significant gaps. The deepfake AI tutor review explicitly identified 'governance and policy gaps' as one of its four major themes, alongside personalization, detection challenges, and ethical implications [2]. The authors proposed a four-pillar governance framework (Transparency and Disclosure, Data Governance and Privacy, Integrity and Detection, and Ethical Oversight and Accountability) precisely because existing policies are insufficient. On the fairness front, the same review flagged algorithmic bias as a key risk, meaning AI tutors could systematically disadvantage certain groups of students based on race, gender, or socioeconomic background. A separate global review of AI tutoring in calculus education (2019–2025) also listed fairness as a persistent issue, along with accuracy problems and lack of teacher involvement [1]. Taken together, these studies suggest that while AI tutoring systems are being rapidly deployed, the governance and fairness safeguards are lagging behind — and the risks are not being fully accounted for in practice.

About These Sources

This answer is built on 4 studies (3 peer-reviewed, 1 preprint) — published from 2025 to 2026, 4 from 2024 or later, 1 in Q1–Q2 journals — selected as the most relevant from 4 studies that passed quality screening, drawn from 41 papers retrieved from a database of over 500 million.

Sources used in this answer

Related research questions