Can audits really keep up with AI that improves every few months?
The short answer is: not easily, and not yet. The most concrete proposal for keeping pace comes from a 2026 paper on Frontier AI Auditing, which defines four AI Assurance Levels (AALs). The authors recommend AAL-1 as a baseline for all frontier AI and AAL-2 as a near-term goal for the most advanced subset of developers [2]. But they also identify four critical requirements for this to work: ensuring high-quality standards so auditing doesn't become a checkbox exercise, growing the ecosystem of audit providers rapidly without sacrificing quality, accelerating adoption by clarifying incentives, and achieving technical readiness for the higher assurance levels [2]. These are not small asks—they represent a major infrastructure build-out that has not yet happened.
A separate 2022 paper on third-party audit ecosystems reinforces this concern by looking at non-AI domains like financial, environmental, and health regulation. It concludes that the institutional design of audits is far from monolithic and that simply turning toward audits is unlikely to achieve actual algorithmic accountability without sustained focus on how those audits are structured and enforced [4]. In other words, even if we had the will, we don't yet have the institutional machinery to make audits keep pace.
What does a good audit look like, and how far is that from reality?
A best-case audit is thorough, independent, and covers the full lifecycle of an AI system. A 2022 paper from psychology researchers proposes a framework for psychological audits that includes 12 crucial components across three categories: components related to the AI model itself (source data, design, development, features, processes, outputs), components related to how the model is presented and understood by developers, affected people, and third parties, and meta-components like cultural context and respect for persons [1]. This is a rigorous ideal—but it's also a framework designed for high-stakes predictive models, not necessarily for the fast-moving frontier models that are doubling capabilities every few months.
The typical-case reality is much messier. A 2024 guide on maintaining and auditing AI compliance programs focuses on practical steps like handling change management, assuring continuity, developing audit controls, and conducting quality checks [3]. While useful, this is a compliance-oriented approach that assumes a stable regulatory environment—not one where the technology itself is evolving faster than the rules. The gap between the best-case frameworks [1][2] and the typical-case compliance checklists [3] is exactly where the pace problem lives.
What would it actually take for audits to keep pace?
The Frontier AI Auditing paper is the most direct about this. It argues that achieving the vision requires four things: (1) high-quality standards so auditing doesn't become a checkbox exercise or lag behind industry changes, (2) rapid growth of the audit provider ecosystem without compromising quality, (3) stronger incentives for adoption, and (4) technical readiness for high AI Assurance Levels [2]. These are all active gaps. The paper also notes that frontier AI audits should not be limited to publicly deployed products but should consider the full range of organization-level risks, including internal deployment, information security, and safety decision-making processes [2]—which makes the task even harder.
The third-party audit ecosystem paper adds a crucial institutional perspective: audits in other domains (finance, health, environment) are not monolithic, and their success depends on careful design of who audits whom, what standards they use, and what enforcement mechanisms exist [4]. Without that institutional scaffolding, audits risk being performative. So the answer to the question is: independent model audits can potentially keep pace, but only if we build the standards, the ecosystem, the incentives, and the technical readiness simultaneously—and we are not there yet.
About These Sources
This answer is built on 4 studies (3 peer-reviewed, 1 preprint) — published from 2022 to 2026, 2 from 2024 or later, 1 in Q1 journals, collectively cited 271 times — selected as the most relevant from 4 studies that passed quality screening, drawn from 53 papers retrieved from a database of over 500 million.
Sources used in this answer
Auditing the AI auditors: A framework for evaluating fairness and bias in high stakes AI predictive models.
Proposes a psychological audit framework with 12 components across three categories (model-related, presentation-related, and meta-components) for evaluating fairness and bias in high-stakes AI predictive models, drawing on over a century of psychological measurement research.
Frontier AI Auditing: Toward Rigorous Third-Party Assessment of Safety and Security Practices at Leading AI Companies
Defines four AI Assurance Levels (AALs) for frontier AI auditing, recommending AAL-1 as a baseline and AAL-2 as a near-term goal for the most advanced systems, but identifies four critical requirements (standards, ecosystem growth, incentives, technical readiness) that are not yet met.
Maintaining and auditing compliance
Provides practical guidance on maintaining and auditing AI law compliance programs, including change management, continuity assurance, audit controls, and employee training, reflecting a compliance-oriented approach to auditing.
Outsider Oversight: Designing a Third Party Audit Ecosystem for AI Governance
Reviews audit systems from financial, environmental, and health regulation and concludes that audits alone are unlikely to achieve algorithmic accountability without sustained focus on institutional design, including who audits whom and what enforcement mechanisms exist.