Do semantic decision engines in network control loops meet their deadlines and coverage claims?

A loop claim needs matched measurement, not just latency

The paper asks which network control loops can admit a semantic decision engine, and which checks the controller must keep. A semantic decision engine such as Jev can return a valid answer and still miss a network deadline, select an infeasible action, or leave the service unverified. A quick answer helps only when the complete path — state collection, communication, decision waiting, control and observation — meets the requested outcome and its deadline.

So the audit separates two things readers often merge. A family can report latency and still not support a loop claim. Support requires measurement matched to the claimed budget and boundary. Of the 139 paper families reviewed, 50 claim their engine fits a control loop or time budget, and 4 report measurement matched to the claimed budget and boundary. Nine families report latency at the 95th percentile or above, and four report deadline attainment. Those are different denominators: reporting a percentile is not the same as showing a deadline was met.

Three axes separate interfaces, execution paths and check owners

The systematization codes each of the 139 families along three axes: decision interface, control and execution path, and check ownership. Interfaces are labelled selection (S), generation (G), deterministic computation (C), or unspecified (U), and a family can carry more than one letter. Check labels distinguish observation state, feasibility or related checks, and candidate coverage; check types with no listed owner are recorded as unresolved.

This coding is what makes the gap visible. Families with a deterministic computation step made 28 of the 50 loop claims, and all four supported claims come from that group. The 72 families without such a step made 22 of the claims, none supported. Only 23 of those 72 name a feasibility owner and 2 name a coverage owner, against 45 and 11 of the 67 families that do have such a step. Check ownership and load evidence concentrate where a deterministic computation step exists.

Independent coding and explicit denominators back the counts

The evidence audit used independent coding with explicit denominators, and the paper reports agreement statistics for the adjudicated categories. Timing endpoint and execution location allow multiple selections, while single-choice rows sum to 405 scopes. For timing endpoint, model return appears in 38 scopes for one coder and 41 for the other, dispatch in 2 and 6, network effect in 11 and 12, service result in 23 and 22, other in 92 and 81, not reported in 251 and 252, and unclear in 7 and 5.

The same table shows how thin some of the underlying evidence is. Deadline attainment is reported in 7 scopes and not reported in 397, with 1 unclear. Input load is reported in 81 scopes and not reported in 324. Queue or waiting is reported in 11 and not reported in 394. Supported loop claims are matched in 7 scopes, mismatched in 1, and insufficient in 397. These denominators matter because a claim about what the literature measures has to be read against the number of scopes coded, not against the number of papers alone.

Bounded tests show each gap can reverse an admission verdict

The paper does not stop at missing evidence. It runs bounded measurements in one event model, with four diagnostic lenses: measurement boundary, workload summary, check placement and robustness scope. The service-orchestration study, the RAN study and the paper's own application tasks become instances of one framework, and fixed-action interventions on transport and edge execution stacks vary only the decision delay. Each test measures the consequence of one gap, and the three studies are compared without pooling their effects.

For missing load evidence, a decision that meets a 10 s budget for every isolated request meets it for none at 4.992 arrivals/s once decisions queue ahead of measured execution times. For unnamed check ownership, an engine that owns the coverage check requests a new candidate for 45 of 48 contracts without one, but escalates only 4 of 120 infeasible routes. The same engine passes one coverage check and fails another. These are bounded tests under one event model, not field deployments, and the paper presents them as demonstrations that a gap can flip a verdict rather than as estimates of how often it does.

The timing endpoint sets the decision's share of elapsed time

The framework separates the queue from the decision itself. In the timing sketch, Q is the wait from arrival to slot start, D is the decision interval from slot start to accepted, and the endpoints run from arrival to ACK and arrival to verified. A decision's share of elapsed time depends on which endpoint is named, so a number reported at model return is not comparable to one reported at a verified outcome.

This is why the audit treats a named timing endpoint as a prerequisite for a loop claim. The paper's own conclusion is that a loop claim needs to state its endpoint, its load and its check ownership. The bounded tests support the load part directly: the endpoint set the decision's share of elapsed time, and load turned a decision that met every isolated deadline into one that met none.

Accuracy degrades as contracts, telemetry and policies grow

The paper also reports how engine accuracy behaves as the structure of the task grows. Contract fields are tested at 4, 6 and 8 output fields; telemetry cells at 3, 7, 21 and 57; policy requirements at 4, 16 and 64. Panels for contract fields and telemetry cells are borrowed and reanalysed from the Edge and RAN studies, while the policy-requirement panel is owned by this paper. Bands show 95% Wilson intervals.

A paired padding experiment adds 300 padding tokens and measures the change against a base condition. For service-contract padding at 16384 tokens, Jev-1.13.0 shows a change of -2.7 accuracy points with a 95% interval of [-5.7, 0.3], while Qwen3.5-4B-JSON shows -30.0 points with an interval of [-35.3, -24.7]. Rule and Oracle show 0.0 change. In the access-policy setting, irrelevant text and repetition are added at 120 tokens; Qwen3.5-4B shows +8.3 points for repetition with an interval of [0.8, 15.8], while the full-rule, cheapest and first baselines show 0.0. The padding panel uses paired stratified-bootstrap intervals, so these are paired contrasts rather than independent-group comparisons.

The audit measures reporting, not whether engines work

The evidence audit measures what the literature reports. A family coded without matched timing evidence may still operate within its claimed loop, so the audit establishes missing support for admission claims rather than proving that those engines fail. The measured consequences come from the Edge and RAN source studies and the paper's own tasks; the surveyed network systems of other authors remain untested.

The bounded experiments also have a narrow statistical base. Each experiment uses one random stream, and its intervals come from resampling within the run, by input, scene or time block, so they do not describe variation across independent replications. Because the synthesis questions and comparison families were formulated after the measurement records were inspected, their tests are exploratory, and Holm adjustment controls error only within each declared family. The access-policy, placement and forwarding tasks are constructed application-layer tasks.

A loop claim should name endpoint, load and check owner

The paper's practical output is a minimum reporting record, design rules and a research agenda for admitting decision engines to control loops, each derived from a gap in the systematization or the audit. The reporting record follows directly from where the evidence was thinnest: deadline attainment reported in 7 of 405 scopes, queue or waiting in 11, and a named coverage owner in only 2 of the 72 families without a deterministic computation step.

For a reader deciding whether to admit an engine to a control loop, the paper's own priority ordering is therefore specific. First check whether the claim names a timing endpoint, because the endpoint sets the decision's share of elapsed time. Then check whether load and queueing are reported, because a decision that meets a 10 s budget for every isolated request met it for none at 4.992 arrivals/s once decisions queued ahead of measured execution times. Then check who owns the coverage check, because an engine that owned it requested a new candidate for 45 of 48 contracts without one while escalating only 4 of 120 infeasible routes. A claim that omits all three is the case the audit found 22 times and supported zero times.

Sources used in this answer