Refining clinical logic, not just wording
The paper targets a narrow but high-value object: the history of present illness (HPI) inside hyperthyroidism admission notes written by residents. The HPI is defined here as the retrospective account ending at admission — onset time, predisposing factors, symptoms, the pre-admission diagnostic and treatment course, reason for admission, and general condition. The forward-looking treatment plan for the current hospitalization is explicitly excluded.
That scope matters because the authors frame the gap as deeper than formatting. Existing documentation tools, in their account, summarize content or correct terminology, grammar, and specific factual errors at the document level, but do not systematically address clinical logic, narrative coherence, or alignment with guidelines. HQ-ANR is therefore positioned as a quality-control model that rewrites the HPI toward a structured, guideline-anchored narrative rather than merely polishing sentences.
How retrieval and agent roles constrain rewriting
HQ-ANR is built on Qwen3-32B, a general-purpose model, and adds three things. First, a structured HPI template derived by parsing expert-written notes and confirmed by clinical experts. Second, a dual-source knowledge base combining global knowledge (Chinese guidelines on thyrotoxicosis and hyperthyroidism, plus textbooks) with local knowledge (in-house expert admission notes); treatment protocols were deliberately left out to avoid altering patients' existing regimens. Third, a multi-agent pipeline: an annotation agent splits the original HPI into modules, symptom and examination agents retrieve diagnosis-driven evidence and rewrite their modules, a treatment agent normalizes medication documentation and inserts placeholders where dosage or frequency is missing, and a composition agent merges the modules and checks cross-paragraph consistency.
The comparison condition is the same base model with chain-of-thought prompting, asked to perform the same optimization task. All outputs from both groups were scored by clinical experts under blinded conditions. The knowledge base and validation set were separated at the patient level, so no patient contributed notes to both.
Quality gains concentrate where baselines were lowest
The Physician Documentation Quality Instrument (PDQI-9) scores nine dimensions from 1 to 5 and sums them. HQ-ANR lifted the median total from 39.00 (IQR 37.13–40.88) to 42.00 (IQR 41.00–43.00), while chain-of-thought prompting reached 41.00 (IQR 39.63–42.38); the between-model difference was significant (P < 0.001).
The distribution of that gain is the more informative result. The authors report that the total-score improvement was driven mainly by "Useful" and "Thorough," the two dimensions with the lowest starting scores, while other dimensions were already near ceiling in the original notes. HQ-ANR raised "Useful" from a median of 2.5 to 4.00 and "Thorough" from 3.25 to 4.00, and also improved "Synthesized" and "Comprehensible." Its "Accurate" score stayed statistically unchanged from the original (P = 0.783), whereas chain-of-thought prompting significantly lowered "Accurate" (P = 0.014).
Two dimensions did not favor the full system. Chain-of-thought outputs scored better on "Succinct" (P = 0.020) and "Internally Consistent" (P = 0.028), and HQ-ANR's "Internally Consistent" score was slightly below the original notes (P = 0.071), which the authors flag as room for improvement in cross-section consistency checks.
What the 6.67% hallucination rate counts
The headline safety number is a note-level rate: the percentage of notes containing at least one positive factual hallucination. HQ-ANR reached 6.67% of the 30 notes, against 26.67% for chain-of-thought prompting. The paper restricts this measure to positive factual hallucinations, so it should not be read as a general error rate covering omissions or stylistic problems.
The chain-of-thought failures are broken down by harm likelihood and severity. Across 10 hallucination instances in 30 notes, 2 were rated low likelihood and no harm (omission of non-critical information), 5 low likelihood and mild (added non-critical hyperthyroidism-positive symptoms, or a missing time point), 1 medium likelihood and mild (non-essential medication doses or drug names), and 2 high likelihood and moderate (added critical medication doses, timeline confusion). Each note is counted once, under its highest harm level. The authors attribute the longer physician refinement time for chain-of-thought outputs to this higher hallucination rate, since errors require more manual verification.
Retrieval suppresses hallucination; agents add depth
The ablation results separate the two components' contributions. Removing RAG increased hallucinations and slightly reduced factual accuracy, and the ablated model's total PDQI-9 score fell from 41.88 to 40.32 (P < 0.001), with significant declines in "Accurate," "Thorough," "Useful," "Succinct," and "Synthesized." Replacing multi-agent decomposition with a single-pass baseline produced only modest improvements — significant in "Useful" but not in "Thorough" — and a low hallucination rate that the authors say reflected a conservative strategy of minimal edits rather than effective retrieval use.
The authors' reading is that RAG is the primary driver of hallucination suppression, since its presence reduced hallucination rates whether or not the multi-agent architecture was present, while the PDQI-9 quality gain required both components. Their interpretation: a powerful rewriting system without external factual anchoring can be clinically unsafe, but properly anchored it can produce substantial improvement. This is the paper's own explanation of the ablation pattern, not an independently tested causal claim.
Refinement time fell by about two minutes per note
Efficiency was measured as physician refinement time on the same 30 notes. Original notes took a mean of 216.3 s (SD 67.1); chain-of-thought outputs took 141.5 s (SD 48.5), saving 74.7 s (95% CI 62.8–87.7); HQ-ANR outputs took 103.3 s (SD 24.9), saving 112.9 s (95% CI 93.2–132.7), a 52.2% efficiency gain. HQ-ANR versus chain-of-thought saved a further 38.2 s (95% CI 26.5–49.9), all with P < 0.001.
These are measured refinement times within the study's evaluation procedure, not observed savings in a live clinical workflow. The authors state that the workflow benefits — freeing senior physicians for complex decisions and giving junior physicians context-sensitive writing assistance — require confirmation in prospective real-world studies.
Where the evidence stops
Validation used single-center retrospective data from one hospital's electronic medical record system, which the authors note may introduce selection bias. Effect sizes come from 30 notes and two raters, with moderate-to-good inter-rater agreement on PDQI-9 totals (ICC 0.650 for original notes, 0.613 for HQ-ANR, 0.674 for chain-of-thought). The authors also note that repeated generations could yield different outputs, particularly for a rare event like a 6.67% hallucination rate, and that they did not quantify this variability.
The scope is one disease and one note section. The authors argue the framework is not inherently disease-specific — updating the knowledge base extends it to other diseases and adjusting templates adapts it to other document types — but the current validation does not establish that transfer. Cases involving thyroid cancer, pituitary lesions, severe systemic disease, or off-label medication use were excluded, so the evaluated population is uncomplicated-to-complicated hyperthyroidism within those bounds.
The comparison that matters is accuracy under rewriting
The paper's most decision-relevant contrast is not the total score but the pairing of quality gain with factual stability. Chain-of-thought prompting improved coverage and usefulness while significantly lowering "Accurate" and producing a 26.67% note-level positive hallucination rate, including two high-likelihood, moderate-severity instances involving critical medication doses and timeline confusion. HQ-ANR improved "Useful" and "Thorough" while holding "Accurate" statistically unchanged and cutting that rate to 6.67%.
For readers weighing this approach, the operative threshold is therefore whether a documentation system can deepen rewriting without loosening factual anchoring — and the ablation suggests the anchoring comes from retrieval, while the depth comes from agent decomposition. The unresolved cost is consistency: HQ-ANR's slight decline in "Internally Consistent" and its weaker "Succinct" score relative to chain-of-thought mark the specific dimensions a follow-up evaluation would need to check before this pipeline is treated as a general documentation quality-control platform.
Sources used in this answer
Quality optimization with low positive factual hallucination for the HPI in hyperthyroidism admission notes using multi agent LLM with RAG
In a blinded evaluation of 30 resident-drafted hyperthyroidism notes, HQ-ANR raised median PDQI-9 total scores from 39.00 to 42.00 and reduced the note-level positive factual hallucination rate to 6.67%, versus 26.67% for chain-of-thought prompting, while saving 112.9 s of physician refinement time per note.