What's the best-case accuracy, and what does that really mean?
The strongest quantitative result comes from a 2025 study introducing ECNU-ChemGPT, a chemistry-specialized LLM that achieved 68.3% top-1 accuracy on the USPTO_50K dataset, a standard benchmark of 50,000 reactions. In plain terms, when asked to propose the correct reactants for a given molecule, it gets the top suggestion right about two out of three times. That's a solid performance—better than general-purpose models like GPT-4o in the same study—but it also means one in three suggestions is wrong, which is a non-trivial failure rate in a field where a wrong route wastes time and resources.
The same study also showed the model successfully reconstructed 13 complete experimental pathways for real drug molecules from medicinal chemistry journals. That's a promising sign for practical utility, but it's a small, curated sample, not a guarantee of everyday reliability. The takeaway: the best models are good enough to be genuinely useful, but not good enough to trust blindly.
Why do chemists still need to stay in the loop?
The evidence consistently points to the need for human guidance. A 2023 study showed that by adding a 'disconnection prompt'—telling the model where to break the molecule—the model's performance improved by 39% over the baseline, and it proposed a broader, more diverse set of precursors. This means the model's default behavior is biased by its training data, and it takes a chemist's input to steer it toward more creative and useful suggestions. Without that input, the model tends to fall back on familiar reaction types, limiting exploration of chemical space.
Another 2023 study reinforced this by showing that a retrosynthesis Transformer could increase prediction diversity by prepending a classification token to the target molecule, allowing chemists to steer the model toward different disconnection strategies. This is a clear signal that these tools are not autonomous problem-solvers; they are interactive assistants that work best when a human sets the direction. The 2026 study on strategy-aware planning takes this further, showing that LLMs can guide search algorithms toward chemically meaningful solutions when chemists specify strategies in natural language—again, the human is the strategist, the model is the executor.
What's the gap between the best-case and typical-case evidence?
The 68.3% accuracy is a best-case figure from a specialized, heavily fine-tuned model. The broader literature, as reviewed in a 2022 survey, notes that AI-driven retrosynthesis still faces challenges like heavy reliance on expertise and sub-optimal routes. The survey doesn't give a single accuracy number, but it highlights that the field is still evolving and that no single approach is a silver bullet.
The 2023 studies, while not reporting overall accuracy, show that even state-of-the-art models at the time had biases that required prompting to overcome. This suggests that typical performance, without careful prompting or human oversight, is likely lower than the best-case 68.3%. In practice, a medicinal chemist might see correct suggestions less often, especially on novel or complex molecules that fall outside the training data's comfort zone. So, the honest answer is: the best models are approaching 'useful assistant' territory, but 'dependable' in the sense of 'trust it without checking' is still out of reach.
About These Sources
This answer is built on 5 studies (4 peer-reviewed, 1 preprint) — published from 2022 to 2026, 2 from 2024 or later, 3 in Q1 journals, collectively cited 149 times — selected as the most relevant from 5 studies that passed quality screening, drawn from 31 papers retrieved from a database of over 500 million.
Sources used in this answer
Chemical reasoning in LLMs unlocks strategy-aware synthesis planning and reaction mechanism elucidation
A 2026 study showed that LLMs, when integrated with Monte Carlo Tree Search, can guide retrosynthetic planning and mechanism elucidation based on chemist-specified strategies, with newer and larger models showing increasingly sophisticated reasoning.
Unbiasing Retrosynthesis Language Models with Disconnection Prompts
A 2023 study found that using disconnection prompts in retrosynthesis language models improved performance by 39% over baseline and increased diversity of precursor suggestions, empowering chemists to control disconnection predictions.
Enhancing diversity in language based models for single-step retrosynthesis
A 2023 study demonstrated that prepending classification tokens to target molecules in a retrosynthesis Transformer model consistently improved prediction diversity, enabling recursive synthesis tools to circumvent dead ends.
A Large Language Model for Chemistry and Retrosynthesis Predictions
A 2025 study introduced ECNU-ChemGPT, a chemistry-specialized LLM that achieved 68.3% top-1 accuracy on USPTO_50K and reconstructed 13 complete experimental pathways for real drug molecules, outperforming general-purpose models like GPT-4o.
Artificial Intelligence for Retrosynthesis Prediction
A 2022 review of AI-driven retrosynthesis prediction outlined the field's challenges, including reliance on expertise, sub-optimal routes, and computational costs, and categorized AI techniques that address these issues.