What minimum sensitivity and specificity do people accept for AI disease detection?

Participants set a high bar for AI detection

The paper asks a simple but rarely quantified question: how much error will patients and the public tolerate from an AI system that detects disease? Instead of asking people to rate a device, the researchers asked them to state the largest number of errors they would accept out of 100 people, then converted those answers into minimum acceptable sensitivity and specificity.

Across 1,328 participants, the mean minimum acceptable sensitivity was 0.90 (SD 0.20) and the mean minimum acceptable specificity was 0.90 (SD 0.19). The medians were higher still, at 0.97 for both measures, with an interquartile range of 0.90 to 1.00. About one third of participants chose the maximum possible value of 1.00: 451 (34.0%) for sensitivity and 443 (33.4%) for specificity. In other words, the typical answer was not a modest tolerance for error but a demand close to perfect performance.

Two error types became two separate thresholds

Sensitivity and specificity are easy to conflate, so the survey separated them at the point of measurement. Participants first entered the maximum acceptable number of missed cases among 100 people who actually had the disease. That count, divided by 100, is the maximum acceptable false-negative fraction, and one minus that fraction is the minimum acceptable sensitivity.

Participants then entered the maximum acceptable number of people incorrectly classified as having disease among 100 people without disease. That count divided by 100 is the maximum acceptable false-positive fraction, and one minus it is the minimum acceptable specificity. Because the two questions were asked separately, the resulting pair does not necessarily describe a single operating point that a device could achieve at once. The authors note this explicitly, and it matters for reading the equal 0.90 means: they do not show that participants weighted missed cases and false alarms equally.

A national sample plus long-term cancer survivors

The main group was a nationwide survey of 1,222 adults aged 19 to 84 in the Republic of Korea, conducted in August 2025 with proportional stratified sampling across all 17 provinces and quotas for age, sex, and region. Face-to-face administration was offered to participants aged 65 or older. A separate cohort of 106 long-term lung cancer survivors was recruited from Samsung Medical Center; eligibility required a confirmed lung cancer diagnosis and either nearing completion of five years of post-treatment follow-up or having completed at least five years and remaining under surveillance.

The hypothetical system was deliberately unspecified. The questions did not name a target condition, clinical setting, intended use, disease prevalence, or whether the AI would work alone or assist a clinician. That design keeps the answers comparable across respondents, but it also means the reported thresholds describe a general expectation rather than a requirement for any particular test. The authors state that the values should not be read as implementation thresholds.

Most participants asked for more than IDx-DR delivers

To give the answers an external reference, the researchers reviewed FDA decision summaries for AI-based devices reporting both sensitivity and specificity. Among the disease-detection devices reviewed, IDx-DR had the highest reported values, with observed sensitivity of 0.874 and specificity of 0.895. Because responses were recorded in increments of 0.01, a minimum acceptable sensitivity of at least 0.88 and specificity of at least 0.90 were counted as exceeding that device's reported performance.

By that rule, 1,084 participants (81.6%) reported minimum acceptable sensitivity of at least 0.88, and 1,083 (81.6%) reported minimum acceptable specificity of at least 0.90. The comparison is device- and indication-specific: the authors stress that FDA authorization does not set a universal minimum, and that cross-device comparisons are limited by differences in target conditions, settings, and validation studies. Still, the pattern is the paper's most decision-relevant result: for many respondents, an authorized device's reported accuracy would fall short of what they consider minimally acceptable.

Cancer history was linked to stricter expectations

In the combined analysis, participants with a cancer history reported higher minimum acceptable sensitivity and specificity after adjustment for age, sex, and chronic disease categories. The adjusted difference was 0.046 (95% CI 0.001 to 0.091) for sensitivity and 0.058 (95% CI 0.014 to 0.102) for specificity. Unadjusted comparisons were also significant, with both p values below 0.001.

The source of that association needs care. Of the 134 participants with a cancer history, 106 came from the separately recruited lung cancer survivor cohort, whose mean minimum acceptable sensitivity was 0.96 (SD 0.12) and specificity 0.97 (SD 0.09). When the analysis was restricted to the nationwide survey alone, where cancer history was self-reported and only 28 participants reported it, cancer history was not clearly associated with either measure. The authors conclude that the combined result appears to reflect, at least in part, the survivor cohort. Within that cohort, values stayed near 1.00 across cancer stage, treatment history, and time since diagnosis, but subgroup differences had wide confidence intervals, and only six participants had stage III-IV disease.

High specificity does not guarantee a useful positive result

The paper also translates the group means into predictive values under assumed disease prevalence from 1% to 20%. At 1% prevalence, the calculated positive predictive value was 7.82% among participants without a cancer history and 16.59% among those with one; negative predictive value exceeded 99% in both groups. As prevalence rose to 20%, positive predictive value increased to 67.73% and 83.11%, respectively, while negative predictive value fell to 97.02% and 98.58%.

These figures are derived from group mean responses and assumed prevalence, not from participant-reported predictive values or observed device performance. Their value is illustrative: they show how strongly the meaning of a positive AI result depends on the population in which the test is used. The authors use this to argue that communication should explain the probability that a positive or negative result reflects true disease in the intended-use population, not just quote sensitivity and specificity.

Where these thresholds stop being a target

Several features limit how far the numbers travel. The scenario was general and hypothetical, and sensitivity and specificity were elicited separately, so the values are expectations rather than a jointly attainable operating point. The lung cancer survivor cohort came from a single tertiary center, while the nationwide survey was nationally representative; the authors caution that the two groups differ in recruitment source and clinical characteristics and should not be treated as a direct comparison.

Demographic differences were present but uneven. Within the nationwide survey, minimum acceptable sensitivity and specificity differed across education levels (p = 0.009 and p < 0.001), and specificity also differed by marital status (p = 0.009) and household income (p = 0.010), while sensitivity did not. Neither measure differed significantly by employment status. These are associations within one survey, not evidence that any of these characteristics causes a particular threshold.

The practical priority is communicating error in context

The paper's most useful takeaway is a comparison, not a universal cutoff. The participants who reported a cancer history were the group most likely to demand near-perfect performance: 91.0% of those with a cancer history reported minimum acceptable sensitivity of at least 0.88 and 92.5% reported specificity of at least 0.90, the highest proportions among the chronic disease categories examined. For clinicians and developers, that split suggests that communication should be tailored to people with prior disease experience, who may judge an authorized device against expectations close to 1.00.

The authors' recommendation follows from the same evidence: explain the expected frequency and consequences of false-negative and false-positive results, and explain how prevalence changes predictive values in the intended-use population. The survey cannot say which threshold a regulator or health system should adopt. It can say that a large share of respondents set minimums above the reported performance of the strongest reference device reviewed, and that this gap is widest among those with a cancer history.

Sources used in this answer