AI in the Dock: Why Explainability is the New Legal Standard for Liability
Explainable AI under contract and tort law: legal incentives and technical challenges
The paper investigates the intersection of AI explainability and legal liability, specifically within contract and tort law. It introduces a framework where professional actors (e.g., in healthcare and corporate sectors) may be legally mandated to adopt explainable ML models to fulfill their "duty of care," while demonstrating the technical trade-offs between accuracy and interpretability.
TL;DR
While the tech world focuses on the GDPR's "right to explanation," a far more potent driver for Explainable AI (XAI) is emerging: Tort and Contract Law. This paper argues that doctors and corporate managers may soon be legally required to use AI to avoid malpractice, but they can only safely do so if those models are explainable. Without interpretability, professionals cannot exercise the "reasoned disagreement" necessary to shield themselves from liability when AI inevitably makes a "non-human" error.
The Motivation: Moving Beyond Data Protection
The legal debate around AI is often stuck in the mud of the GDPR’s Recital 71. Is there a "right to explanation"? Maybe, maybe not.
The authors shift the focus to a more practical arena: Liability. In professions governed by a "standard of care," failing to use a tool that is statistically proven to outperform humans (like an AI that detects cancer 75 months earlier than clinicians) could be seen as negligence. However, using a black box that provides no "why" prevents professionals from catching "Clever Hans" predictors—models that look accurate but are actually picking up on noise or bias.
Methodology: The Legal-Technical Bridge
The paper employs a dual-track approach:
- Legal Case Studies: Analyzing Medical Malpractice and Mergers & Acquisitions (M&A) to determine when the "Standard of Care" shifts to mandate AI adoption.
- Technical Mapping: Categorizing ML models by their "Transparency" (simulatability) vs. "Post-hoc Interpretability" (local explanations).
The Model-Explanation Matrix
The authors provide a crucial taxonomy of how different models meet the need for "Meaningful Information."
The Core Concept: The "Human-Machine Team"
The authors posit that the law doesn't want humans to be mere "executors" of AI decisions. To avoid liability, a doctor must be able to:
- Identify Outliers: Recognize when a patient doesn't fit the training data.
- Exercise Reasoned Disagreement: Override the AI based on professional knowledge without the override itself being seen as negligence.
This is only possible if the AI is explainable. If a model is a total black box, the human user has no basis for disagreement, leading to "automation bias" and increased legal risk.
Experimental Evidence: The Spam Case Study
To test the "Accuracy-Explainability Trade-off," the authors ran a spam detection experiment.
Key Insight: In many tasks, simple, highly explainable models (like Naive Bayes or Logistic Regression) perform almost as well as deep neural networks. In these "low-complexity" domains, the legal standard should arguably mandate the simpler, explainable model because the marginal gain in accuracy for a CNN (0.985 vs 0.977) does not justify the loss of transparency.
Critical Analysis & Conclusion
The Takeaway
Explainability is a pre-condition for the duty to use AI. While it might be legitimate to use a black box if it's exceptionally accurate, a professional cannot be obligated to use a tool they cannot critically assess.
Limitations
The paper acknowledges that for "supra-human" tasks (like complex image recognition), the accuracy-explainability trade-off is real and painful. The law may eventually have to accept "validation-based trust" (trusting the model because it has worked 99.9% of the time) rather than "reason-based trust."
Future Outlook
As AI moves from "experimental" to "standard of care," the tech industry must pivot. We need "Contractual Explainability"—interfaces designed not just for engineers, but for doctors and managers to fulfill their legal duties. The future of AI deployment isn't just about higher F1 scores; it's about building the "paper trail" of reasoning that stands up in court.