Why less data disclosure makes audits harder — and what the evidence shows
When model providers share less data, auditors lose the ability to directly test for bias, accuracy, or safety on real user information. A 2024 taxonomy of AI audit practices found that only a subset of audits translate to desired accountability outcomes, and limited data access is a major barrier [2]. The study categorizes audits by who does them — regulators, law firms, civil society, journalists, academics, and consultants — and finds that those with less access to internal model data (like journalists or civil society groups) often produce weaker accountability results [2]. This means that without sufficient data, even well-intentioned audits may fail to catch problems or force changes.
The practical impact is that auditors must rely on what they can get: public APIs, limited test sets, or synthetic data. This restricts the depth of analysis. For example, a fairness audit that cannot access protected attributes (like race or gender) in the original training data may miss systematic biases. The 2024 paper [2] emphasizes that audit design and institutional context — including data access — are critical for effectiveness.
Can synthetic data fill the gap? A 2025 framework shows promise
One workaround is to use synthetic data that mimics the original dataset without exposing sensitive information. A 2025 study proposed a framework that generates differentially private synthetic data — data altered just enough to protect privacy while preserving statistical patterns [1]. In experiments on three real datasets (Adult, COMPAS, and Diabetes), the researchers compared fairness metrics from synthetic data to those from real data. They found that the synthetic data preserved fairness properties closely enough to enable meaningful evaluations, though exact numbers varied by dataset and metric [1]. For instance, on the COMPAS recidivism dataset, the difference in fairness metrics between synthetic and real data was small enough to still detect bias patterns.
This means that even when a provider refuses to share raw data, an auditor could request a differentially private synthetic version and still conduct a useful fairness audit. The trade-off is a slight loss in precision — the synthetic data is not perfect, but it is far better than no data at all. The 2025 paper [1] demonstrates that this approach balances rigorous auditing with strong privacy protections, making it applicable in sensitive domains like criminal justice or healthcare.
A three-layered approach: auditing at multiple levels to compensate for data limits
For large language models (LLMs), which are often black boxes with limited disclosure, a 2023 paper proposes a three-layered audit strategy: governance audits of the provider, model audits after training but before release, and application audits of the final product [3]. This layered approach means that even if the provider discloses little about the model's internals, auditors can still check the provider's policies (governance layer) and test the application's behavior (application layer). The model layer — which requires the most data — can be supplemented by the other two layers.
The 2023 paper [3] explicitly acknowledges that auditing LLMs is challenging due to emergent capabilities and wide downstream uses, but argues that structured, coordinated audits across all three levels can identify and manage ethical and social risks. However, it also cautions that auditing alone cannot solve all governance problems — it is a tool, not a panacea. This suggests that when data disclosure is minimal, a multi-level audit strategy can still catch issues, but the auditor must be realistic about what can be achieved.
About These Sources
This answer is built on 3 studies (2 peer-reviewed, 1 preprint) — published from 2023 to 2025, 2 from 2024 or later, collectively cited 206 times — selected as the most relevant from 3 studies that passed quality screening, drawn from 33 papers retrieved from a database of over 500 million.
Sources used in this answer
Quantitative Auditing of AI Fairness with Differentially Private Synthetic Data
A 2025 framework using differentially private synthetic data on three real datasets (Adult, COMPAS, Diabetes) showed that fairness metrics from synthetic data closely align with those from real data, enabling meaningful audits while protecting privacy.
AI auditing: The Broken Bus on the Road to AI Accountability
A 2024 taxonomy of AI audit practices found that only a subset of audits lead to accountability outcomes, with limited data access being a key factor; audit design and institutional context are critical for effectiveness.
Auditing large language models: a three-layered approach
A 2023 paper proposes a three-layered audit approach for large language models (governance, model, application) to compensate for data limitations, but acknowledges that auditing alone cannot solve all governance challenges.