ARA: Moving Beyond the PDF to an Agent-Native Scientific Commons

TL;DR

The traditional scientific paper is a "lossy compression" of the research process, optimized for human reading but inadequate for AI agents. The Agent-Native Research Artifact (ARA) protocol replaces linear PDFs with structured, executable knowledge packages. By preserving failed experiments and providing precise code specifications, ARA enables agents to understand papers with 93.7% accuracy and significantly improves the success rate of autonomous reproduction and extension.

Background: The Hidden Taxes of Research

In the current academic ecosystem, research is a branching tree of hypotheses, but the published paper is a single pruned branch. The authors identify two primary inhibitors of progress in the age of AI:

1. The Storytelling Tax: The systematic erasure of "negative knowledge." When we discard failed experiments to fit a narrative, we force future researchers (and agents) to rediscover the same dead ends at great token and compute expense.
2. The Engineering Tax: The massive gap between what a human reviewer needs to believe a claim and what an agent needs to execute it. Only 45.4% of reproduction requirements are fully specified in modern ML papers.

Methodology: The Four-Layer Architecture

ARA abandons the document-centric view in favor of an executable file-system ontology.

1. The ARA Protocol

The artifact is organized into four interlocking layers designed for "progressive disclosure," allowing agents to load only what they need:

• Cognitive Layer (/logic): Maps the scientific logic—observations, gaps, and falsifiable claims.
• Physical Layer (/src): Contains the code kernel or full repository, annotated for execution.
• Trace Layer (/trace): A "Git log for research" that preserves the branching DAG of decisions, pivots, and failures.
• Evidence Layer (/evidence): Raw metrics and logs that ground every claim.

Figure: Cross-layer structure of a real ARA. Claims in /logic link to /src and /evidence via forensic bindings, while the /trace preserves failure nodes (marked ×).

2. Live Research Manager & Compiler

The authors realized that researchers shouldn't have to author these files manually. The Live Research Manager works as an agent skill, distilling natural language conversations between researchers and AI assistants into structured ARA events in real-time. For existing literature, the ARA Compiler performs "forensic reconstruction" to decompress PDFs and repos back into structured layers.

Experiments and Results

The evaluation focused on three tiers of utility: Understanding, Reproduction, and Extension.

1. Knowledge Extraction

On a corpus of 450 questions across 30 targets, ARA increased accuracy from 72.4% to 93.7%. The most significant jump came from Category C (failure knowledge), where baseline PDFs were effectively "blind" to why certain methods were abandoned.

2. Reproduction Success

Using the PaperBench benchmark, the ARA protocol improved difficult-task reproduction from 46.0% to 54.5%. By providing a "Physical Layer" with explicit configurations, agents spent less time debugging environments and more time verifying scientific claims.

Figure: Performance across easy, medium, and hard tasks showing the widening gap as complexity increases.

3. The Power of "Negative Knowledge"

In RE-Bench extension tasks, agents were asked to beat a reference score. ARA agents reached useful first moves faster than "Paper" agents by consulting the failure traces. However, a fascinating nuance emerged: for highly capable models (like Claude 4.6), the recorded failure traces occasionally acted as a "cognitive anchor," preventing the agent from discovering creative solutions that were not present in the original researcher's playbook.

The ARA Seal: A New Era of Peer Review

The authors propose a "CI/CD for Science." Before a human ever reads a paper, the ARA Seal validates it:

• Level 1: Structural integrity (Does the code link to claims?).
• Level 2: Argumentative rigor (Is the methodology sound?).
• Level 3: Execution (Do the results actually reproduce?). This allows human reviewers to focus on significance, novelty, and taste, delegating the mechanical "fact-checking" to AI.

Critical Insight: The (Human+AI)² Network

The ultimate vision is a research network where every contribution is an "executable diff." Instead of reading a static 8-page document, a researcher can fork an ARA, extend a claim, and merge the result back into the scientific commons.

Limitations

• ML-Centric: Currently optimized for computational sciences; extension to "wet-lab" biology or materials science (physical execution) is future work.
• Fidelity Ceiling: An ARA is only as good as the agent capturing the session or the data available in the original PDF.

Conclusion

The "The Last Human-Written Paper" argues that as AI agents become the primary consumers of research, the formats we use to communicate must evolve. By treating scientific knowledge as an operable artifact rather than a linear story, we can finally collect the "interest" on the billions of dollars currently wasted on undocumented research failures.