What can LLM research agents actually do today?
They can execute the entire research workflow autonomously. The Agent Laboratory system, for example, takes a human-provided idea and runs it through three stages—literature review, experimentation, and report writing—producing a code repository and a research paper [1]. When powered by a top-tier LLM (o1-preview), its generated machine learning code achieved state-of-the-art performance compared to existing methods [1]. This means a researcher can hand off the labor-intensive parts of a project and focus on creative ideation.
They can also handle specialized, data-heavy tasks in medicine. AI-HOPE lets clinicians ask research questions in plain English, then autonomously analyzes clinical and genomic data to find statistically significant associations—for instance, it independently discovered that TP53 mutations are enriched in late-stage colorectal cancer and that KRAS mutations predict poorer survival in FOLFOX-treated patients, both findings that align with established literature [3]. This removes programming barriers for domain experts.
How much better are they than existing methods?
On standardized benchmarks, the best agents already beat strong baselines by wide margins. The AIRA_2 system achieved a mean Percentile Rank of 81.5% at 24 hours on MLE-bench-30, compared to the strongest baseline's 72.7%—a nearly 11-point improvement [2]. It also exceeded human state-of-the-art on 6 out of 20 diverse research tasks in another benchmark [2]. These gains come from architectural improvements: asynchronous multi-GPU execution, a reliable evaluation protocol, and agents that can dynamically debug their own actions [2].
But the picture is mixed. In the ResearchArena benchmark, which tests an agent's ability to collect and organize information for a research survey, current LLM-based approaches actually underperformed compared to simpler keyword-based retrieval methods [5]. Only the latest reasoning models (like DeepSeek-R1) showed slightly better zero-shot performance [5]. This tells us that while agents excel at executing defined tasks, they still struggle with the open-ended, exploratory phase of research—finding the right papers in the first place.
What are the caveats and risks?
Human involvement still matters a lot. In Agent Laboratory, providing human feedback at each stage significantly improved the overall quality of the final paper [1]. The system is designed as a research assistant, not a replacement—it reduces expenses by 84% compared to prior autonomous methods, but it works best when a human guides it [1].
There are also structural bottlenecks that limit performance. Early agents suffered from three problems: single-GPU execution limited throughput, validation-based selection caused overfitting over long searches, and fixed single-turn LLM operators capped performance [2]. The AIRA_2 system solved these with specific architectural choices, but the fact that such fixes were needed shows that current agents are not plug-and-play [2].
Ethical concerns are real and unresolved. A review of agentic AI in ophthalmology research flags issues of authorship attribution (who gets credit when an agent writes a paper?), data privacy, bias, and accountability [4]. The same review notes that AI-generated manuscripts have already passed peer review, raising questions about the integrity of the scientific record [4]. Clear governance frameworks and validation standards will be essential before these tools can be widely trusted.
About These Sources
This answer is built on 5 peer-reviewed studies — published from 2025 to 2026, 5 from 2024 or later — selected as the most relevant from 6 studies that passed quality screening, drawn from 50 papers retrieved from a database of over 500 million.
Sources used in this answer
Agent Laboratory: Using LLM Agents as Research Assistants
Agent Laboratory, an autonomous LLM framework, completes the full research process (literature review, experimentation, report writing) and reduces research expenses by 84% compared to prior autonomous methods; human feedback at each stage significantly improves output quality.
AIRA_2: Overcoming Bottlenecks in AI Research Agents
AIRA_2 addresses three structural bottlenecks in AI research agents (synchronous execution, generalization gap, fixed operators) and achieves a mean Percentile Rank of 81.5% on MLE-bench-30, outperforming the strongest baseline (72.7%), and exceeds human state-of-the-art on 6 of 20 research tasks.
AI-HOPE: an AI-driven conversational agent for enhanced clinical and genomic data integration in precision medicine research.
AI-HOPE is an LLM-powered conversational agent that autonomously integrates clinical and genomic data for precision medicine; it independently replicated known findings (TP53 mutation enrichment in late-stage colorectal cancer, KRAS mutation association with poorer survival) without prior assumptions.
Toward autonomous discovery: agentic AI and the future of ophthalmic research.
A review of agentic AI in ophthalmology finds that these systems can autonomously perform peer review, hypothesis generation, systematic reviews, and experimental design, but raises ethical concerns about authorship, data privacy, bias, and accountability.
ResearchArena: Benchmarking Large Language Models' Ability to Collect and Organize Information as Research Agents
ResearchArena benchmarks LLMs on academic survey tasks (information discovery, selection, organization) and finds that current LLM-based approaches underperform simple keyword-based retrieval, though reasoning models like DeepSeek-R1 show slightly better zero-shot performance.