KISS: Bridging the Gap Between AI Agents and Process-Based Earth Science
KISS - Knowledge Infrastructure for Scientific Simulation: A Scaffolding for Agentic Earth Science
This paper introduces KISS (Knowledge Infrastructure for Scientific Simulation), a framework that enables LLM agents to execute complex Earth-science process-based models. By externalizing domain expertise into a three-layer "scaffold" (Operators, Protocols, and Recovery Mechanisms), the authors achieved an 84% success rate in end-to-end simulations across 119 models, whereas unassisted agents failed to exceed 40%.
TL;DR
Researchers have developed KISS (Knowledge Infrastructure for Scientific Simulation), a framework that allows AI agents to act as expert "modelers." By providing agents with a structured scaffold of procedural and physical constraints, they increased the success rate of complex scientific simulations from under 40% to over 84%. This work successfully "dissected" 119 different Earth-science models, proving that the expertise required to run these models is not ad hoc but follows a reproducible, universal structure.
The "Operational Barrier" in Scientific Research
For decades, Earth-system science has relied on process-based models—massive codebases that represent our best physical understanding of hydrology, climate, and agriculture. However, these models are notoriously difficult to use.
The problem isn't just about writing code; it's about the tacit knowledge required to make a model "scientifically valid." Which unit should the precipitation data be in? Is a soil moisture value of -0.5 physically possible? When the model crashes with a segmentation fault, is it a coordinate mismatch or a parameter error?
Current LLM agents, despite their coding prowess, often fall into the "Plausible Hallucination" trap: they generate scripts that run without error but produce physically nonsensical results (e.g., negative river flow).
The Methodology: Externalizing Expertise into "KI"
The core insight of the paper is that domain expertise can be externalized into a Knowledge Infrastructure (KI). Instead of asking an agent to "figure it out" from scratch, KISS provides a three-layer scaffold:
- Validated Modeling Operators: Pre-written, validated tools for common tasks (e.g., unit conversion, grid generation).
- Staged Domain Protocols: "Gatekeepers" that check if intermediate results are physically plausible (e.g., mass balance checks).
- Diagnostic Recovery Mechanisms: A "troubleshooting manual" that maps specific error symptoms to validated remedies.
Above: The KDT (Knowledge Dissection Toolkit) extracts structured expertise from messy documentation and source code to create an agent-actionable KI.
Experiments & Results: From Depth to Scale
The researchers tested KISS across two dimensions: Depth (how reliable is it for one model?) and Scale (does it work for many models?).
The Depth Test: The 3,000-Trial Benchmark
Using a coupled hydrology workflow (VIC + Lohmann), the team tested 10 different AI agents (including Claude, GPT, and Gemini).
- With KI: Top-tier agents like Claude 4.5 hit an 84% success rate, completing all 14 milestones of a complex simulation pipeline.
- Without KI (Ablation): Performance collapsed. Agents either fabricated results or looped infinitely on simple errors.
The Scale Test: 119 Models Across 14 Domains
The authors created the Knowledge Dissection Toolkit (KDT) to autonomously generate KI packages. They scaled this to 119 models, covering everything from crop phenology to glacier mass balance.
Surprisingly, they found a structural convergence: across all 119 models, roughly 55% of all failures were related to unit conversions and I/O formats, and 55% of all decisions involved parameter selection and physics options. This proves that "expert modeling" follows a universal pattern regardless of the specific physics involved.
Above: End-to-end validation results show high performance (NSE/r/PBIAS) even for models dissected fully autonomously.
Deep Insights: Why This Matters
The KISS framework represents a "Third Way" in AI-science coupling. We are no longer just replacing physics with neural networks (which lack interpretability) or embedding constraints into loss functions (which is mathematically rigid).
Instead, we are agentizing the existing scientific commons. This has two massive implications:
- Democratizing Science: A farmer in the Mekong Delta could interact with a sophisticated agronomic model via natural language to determine the best rice-sowing date.
- Accelerating Intercomparison: Researchers can now spin up an "ensemble" of 12 different models in a single day—a task that previously required years of international community coordination.
Conclusion & Future Work
The paper concludes that Knowledge Infrastructure is a stronger predictor of success than agent capability. The limitation is no longer the "intelligence" of the LLM, but the "structure" of the knowledge we provide it.
While limitations remain—such as the need for more domain-expert review of autonomous KDT packages—the open-sourcing of HydroCraft and the KDT provides a protocol for the scientific community to build a "living scientific commons."
Disclaimer: This summary is based on the paper "KISS – Knowledge Infrastructure for Scientific Simulation: A Scaffolding for Agentic Earth Science". All metrics and figures are sourced from the original text.