Why detectors fail on new deepfakes they haven't seen before
The core problem is that deepfake detectors are trained on specific generation methods, but new generators appear constantly. A 2024 study showed that detectors trained on one set of deepfakes often fail to catch deepfakes made by a different model, because each generator leaves its own unique artifacts [5]. This is called the generalization gap. Another 2022 experiment compared two leading architectures: a Vision Transformer and a CNN (EfficientNetV2). The CNN specialized on the training methods, but the Vision Transformer showed better generalization to unseen generators, though neither was perfect [7]. The takeaway: even state-of-the-art detectors are brittle when faced with a new generation technique.
Data augmentation can help, but it's a patch, not a cure. One 2023 study used autoencoders to generate synthetic deepfakes for training, which improved generalization to unseen datasets [6]. A 2024 follow-up used adversarial attacks to create new deepfakes from real videos, again boosting generalization without needing new data [5]. These methods show promise, but they are reactive—they try to anticipate what attackers might do, rather than fundamentally solving the detection problem.
Even humans can't reliably spot deepfakes
It's not just machines that struggle—people are poor detectors too. A 2023 study with 529 participants found that listeners correctly identified speech deepfakes only 73% of the time, and giving them examples of deepfakes beforehand only slightly improved performance [4]. The researchers concluded that as synthesis algorithms improve, detection will only get harder. This confirms that deepfakes are a serious security threat because they can fool both automated systems and human judgment.
Political deepfakes are especially dangerous. A 2022 study found that while people with higher analytical thinking skills were slightly better at detecting political deepfakes, overall detection was far from perfect [8]. The study also showed that simply being shown a deepfake video could increase belief in a related fake news story, even if the video itself was flagged as fake. This highlights that the damage is done even when detection is possible.
The arms race: attackers are winning, but defenders are adapting
The pace of deepfake generation is outstripping detection. A 2024 systematic review noted that detection methods are often 'fragmented, reactive, and unable to keep pace' with generative AI advances [2]. Another 2025 review echoed this, calling for 'robust, adaptable detection systems' that can keep up [3]. The consensus is clear: current detection is playing catch-up.
Promising new approaches are emerging, but they are not yet widespread. One 2025 paper proposed a hybrid neural network that achieved 94.3% accuracy on a dataset of 10,000 samples, using ensemble learning and biological signal analysis [2]. Another 2025 study used EEG brain signals to detect deepfakes, achieving 61-62% accuracy—better than chance, but far from reliable [1]. These are proof-of-concept, not production-ready solutions. The field is moving toward multi-modal and adaptive methods, but the attackers have a head start.
Sources used in this answer
Detecting Deepfakes with Super-Resolution EEG.
A deep autoencoder improved EEG resolution for deepfake detection, but accuracy was only 61-62%, barely above chance.
Advancing Techniques for Deepfake Detection and Evaluation: Challenges and Innovations
A hybrid neural network achieved 94.3% accuracy on a 10,000-sample dataset, but the authors note detection is still reactive and fragmented.
Generative Artificial Intelligence and the Evolving Challenge of Deepfake Detection: A Systematic Analysis
A 2025 review found that detection systems lack real-time performance and interpretability, and need to adapt faster.
Warning: Humans cannot reliably detect speech deepfakes.
Humans detected speech deepfakes only 73% of the time in a study of 529 participants, and training only slightly improved performance.
Improving Generalization in Deepfake Detection via Augmentation with Recurrent Adversarial Attacks
Using recurrent adversarial attacks to generate new deepfakes from real videos improved detector generalization without new data.
Autoencoder-based Data Augmentation for Deepfake Detection
Autoencoder-based data augmentation improved deepfake detector generalization to unseen datasets like CelebDF and DFDC Preview.
Cross-Forgery Analysis of Vision Transformers and CNNs for Deepfake Image Detection
Vision Transformers generalized better than CNNs to unseen deepfake generation methods, but neither was perfect.
The detection of political deepfakes
Analytical thinking and political interest helped people detect political deepfakes, but overall detection was far from perfect.