Where do long-context models already work in science?
In genomics, long-context models are already producing practically useful results. The megaDNA model, trained on bacteriophage genomes, can generate de novo sequences up to 96,000 base pairs that contain functional regulatory elements and annotated proteins [2]. This means the model can design entire viral genomes from scratch, a capability with direct applications in synthetic biology and phage therapy. Similarly, PlantCAD2, a plant-specific DNA language model with an 8,192-base-pair context window, improved prediction of accessible chromatin regions in large maize genomes from an AUPRC of 0.587 to 0.711 — a substantial gain that shows long-range context is critical for understanding gene regulation in complex plant genomes [3]. In fact, PlantCAD2 outperformed the much larger Evo2 model (7 billion parameters) on 10 out of 12 zero-shot tasks, demonstrating that a well-designed long-context model can be both efficient and powerful [3].
In software engineering, long-context models are also showing promise but with clear limits. A benchmark called LongCodeU tested 9 popular models on tasks like understanding relationships between code units and long code documentation. The key finding: model performance drops dramatically when code length exceeds 32,000 tokens, far short of the 128K to 1M context windows these models claim [5]. So while long-context models can handle realistic codebases, their practical ceiling is much lower than advertised — a critical caveat for anyone considering them for real software engineering work.
Why aren't they ready for clinical use?
The gap between promise and proof is widest in medicine. A scoping review of 30 original studies on clinical text summarization found that 93% used real patient data, but the research was narrow: 57% focused on radiology reports, 50% used data from intensive care units, and 73% came from US institutions [1]. This means we don't know how well these models would work for other medical specialties, other countries, or other types of clinical text. More concerning, only 7% of studies performed external validation (testing on data from a different hospital or system), only 20% did failure analysis, and just 3% analyzed patient safety risks. None assessed bias [1]. Without these checks, a model that looks accurate on paper could fail dangerously when deployed — for example, by missing a critical finding in a discharge summary from a different hospital system.
The 'lost-in-the-middle' problem is another practical barrier. When a retrieval-augmented generation (RAG) system pulls in multiple documents to answer a medical question, the model tends to ignore information in the middle of the context window. A 2025 study proposed a map-reduce strategy called BriefContext to fix this without retraining the model, and it improved performance across multiple question-answering datasets [4]. This shows that even when long-context models are used, the way information is arranged matters enormously — and that practical solutions are still being developed to make them reliable enough for tasks like clinical decision support.
What would make long-context models practically useful in science?
The evidence points to three concrete improvements. First, evaluation frameworks need to be much more rigorous. The clinical review found that 33% of studies used only automated metrics, which can miss clinically important errors [1]. For genomics, the PlantCAD2 study showed that fine-tuning on specific tasks (like predicting gene expression) can dramatically improve performance, but this requires high-quality labeled data that isn't available for most species [3]. Second, context windows need to be honestly benchmarked. The LongCodeU study showed that models claiming 128K-token context windows actually fail beyond 32K tokens on real code tasks [5]. Users need realistic performance curves, not theoretical maximums. Third, safety and fairness evaluations must become standard. The clinical review found zero bias assessments across 30 studies [1] — an unacceptable gap for any technology that could affect patient care. Until these gaps are closed, long-context models will remain powerful research tools rather than trusted scientific instruments.
About These Sources
This answer is built on 5 peer-reviewed studies — published from 2024 to 2026, 5 from 2024 or later, 2 in Q1 journals, collectively cited 67 times — selected as the most relevant from 5 studies that passed quality screening, drawn from 38 papers retrieved from a database of over 500 million.
Sources used in this answer
Scientific Evidence for Clinical Text Summarization Using Large Language Models: Scoping Review.
A scoping review of 30 studies on clinical text summarization found that only 7% performed external validation, 3% analyzed patient safety risks, and none assessed bias — concluding that current research is exploratory and lacks the reliability needed for clinical use [1].
A long-context language model for deciphering and generating bacteriophage genomes
The megaDNA model, a long-context transformer trained on bacteriophage genomes, can generate de novo sequences up to 96,000 base pairs that contain functional regulatory elements and annotated proteins, demonstrating practical utility in genomics [2].
PlantCAD2: A Long-Context DNA Language Model for Cross-Species Functional Annotation in Angiosperms.
PlantCAD2, a 676-million-parameter plant DNA language model with an 8,192-base-pair context window, outperformed the 7-billion-parameter Evo2 model on 10 of 12 zero-shot tasks and improved accessible chromatin prediction in maize from AUPRC 0.587 to 0.711 [3].
Leveraging long context in retrieval augmented language models for medical question answering
A map-reduce strategy called BriefContext was shown to mitigate the 'lost-in-the-middle' problem in retrieval-augmented generation for medical question answering, improving reliability across multiple datasets without modifying model weights [4].
Benchmarking Long-Context Language Models on Long Code Understanding
A benchmark of 9 long-context language models on code understanding found that performance drops dramatically when code length exceeds 32,000 tokens, far below the 128K to 1M context windows these models claim [5].