The Last Human-Written Paper: Moving from Narrative to Agent-Native Research Artifacts (ARA)

TL;DR

The traditional scientific paper is a lossy compression format. To solve this, The Last Human-Written Paper introduces the Agent-Native Research Artifact (ARA)—a structured, executable protocol that replaces linear PDFs with a four-layer knowledge package (Logic, Code, Trace, and Evidence). By preserving failed experiments and deep configurations, ARA enables AI agents to reproduce research with significantly higher fidelity and accelerate the discovery of new findings.

Background: The Structural Taxes of Science

For centuries, we have published research as stories. While this linear narrative is convenient for human readers, it is fundamentally incompatible with the emerging era of AI-driven science. The authors identify two critical "taxes" imposed by current publication standards:

1. The Storytelling Tax: Authors discard "failed" experiments and rejected hypotheses to maintain a clean narrative arc. Yet, in large-scale agent runs, 90.2% of the total dollar cost is spent on these failed explorations. By throwing them away, we force every subsequent researcher to waste money rediscovering the same dead ends.
2. The Engineering Tax: There is a massive gap between "reviewer-sufficient" prose and "agent-sufficient" specifications. Analyzing 23 ICML papers, the authors found that only 45.4% of reproduction requirements were fully specified in the PDF, even when code was provided.

Methodology: The Four Layers of ARA

The ARA protocol transforms a repository into a "machine-executable knowledge package." Instead of parsing ambiguous prose, an AI agent navigates a strictly defined file-system ontology:

• Cognitive Layer (/logic): Houses structured claims, mathematical formulations, and typed dependency graphs.
• Physical Layer (/src): Contains either a minimal "code kernel" or a manifested repository that connects specific functions to scientific claims.
• Exploration Graph (/trace): The "Git log for research." It preserves the full Directed Acyclic Graph (DAG) of decisions, including dead_end nodes that document why an approach failed.
• Evidence Layer (/evidence): Raw data points and logs that ground every claim.

Capturing Knowledge as a Byproduct

A key innovation is the Live Research Manager. Researchers don't have to manually write these artifacts; instead, a background agent distills researcher-AI conversations into structured updates. This ensures that the "Negative Knowledge" (the failures) is captured in real-time without adding administrative burden to the scientist.

Experimental Results: Turning Understanding into Action

The authors put ARA to the test against traditional PDF+Repo baselines across three tiers: Understanding, Reproduction, and Extension.

1. Understanding: ARA raised Q&A accuracy by 21.3% overall. Crucially, on questions regarding "Failure Knowledge," ARA scored 81.4% while the baseline scored a dismal 15.7% because the information simply wasn't in the PDF.
2. Reproduction: On complex multi-stage pipelines (like PINNs or RL agents), ARA's weighted success rate was 64.4% vs. 57.4%. The advantage grew as the tasks became "Hard," where traditional PDFs omit the most hyperparameters.
3. Extension: By knowing what not to do, agents using ARA reached useful results faster. However, a fascinating "Creative Constraint" emerged: highly capable agents (like Claude 3.7) sometimes performed better without the trace because they weren't anchored by the previous researcher's playbook.

Critical Insight: The (Human+AI)² Network

The endgame of this research is a shift in the scientific ecosystem. Instead of a publishing "event," science becomes a continuous, queryable commons.

• Agent-Native Review: Human reviewers stop checking if numbers match tables (machines do that via ARA Seal Level 1 & 2) and focus instead on novelty, significance, and taste.
• Continuous Maintenance: As software dependencies evolve, agents can use the ARA structure to "self-repair" old research, ensuring it remains executable for decades.

Conclusion & Future Outlook

While currently optimized for CS and Machine Learning, the ARA protocol sets a precedent for "Machine-Readable Science." Its primary limitation is the requirement for an AI-native workflow; however, as specialized coding agents become ubiquitous, the cost of generating these high-fidelity artifacts will trend toward zero. This work effectively argues that the "Last Human-Written Paper" won't be a PDF, but a node in an executable global graph of knowledge.