How Accurate Is Trade Direction Measurement in Polymarket Prediction Markets?

The feed reports the settling taker's side

The paper asks a narrow measurement question: when Polymarket's public trade tape says a trade was a buy or a sell, is that side the side of the order that actually initiated the trade? The test uses the transaction hash carried by each print to locate the settlement of that same trade on-chain, then compares the print with the taker leg of that settlement.

On twelve selected days between April and August 2026, spanning both exchange generations, 92.1% to 100.0% of prints match a settled taker leg under their own hash. On the 24.5 million matched pairs, the side and the token never differ. Size agrees to 10^-6 on 99.9% of pairs, and price agrees to half a cent on every pair, which the paper attributes to rounding to the cent.

The eight days from May to August are the cleanest: 99.8% to 100.0% of prints match. The four April days, which straddle the migration from the first-generation to the second-generation exchange, range from 92.1% to 96.8%. On the two pre-migration days the fill event carries no side field, so the side is derived from which asset the order's maker gave, and the agreement is the same.

Mint-and-merge settlement makes the taker leg essential

Polymarket settles most taker orders by minting or merging complete sets rather than by matching against a resting order on the same token. By count, 53.5% to 73.8% of settled transactions are mints, 17.9% to 27.1% are complementary matches, 1.8% to 3.7% are merges, and 4.2% to 8.1% are mixed.

That structure matters for anyone building a direction benchmark. In a mint, every leg is a buy, and the maker legs carry the taker's side on the complement token. Pooling all settlement legs by token therefore counts a mint's makers as buyers of the other outcome and overstates the buy share. The buy share of taker legs is 80.6% to 85.6%; over all legs it is 84.7% to 88.4%.

The same pooling problem shows up in cell-level agreement. Pooling every settlement leg in an asset-by-five-second cell and comparing the pooled sign with the feed gives agreement on 86.8% to 90.9% of cells, below a majority baseline of 0.906 to 0.947. Against taker legs, agreement is 93.1% to 97.8% of cells, and fill by fill the feed's side agrees on 100%. The paper notes that remaining cell disagreements are not evidence about the feed's side, because feed cells use receipt times and settlement cells use block times.

Four rules, one common scored sample

The paper compares four ways of inferring trade direction. Lee-Ready signs a print above the prevailing midpoint as a buy, below it as a sell, and at the midpoint by the tick test. The quote rule signs a print at or above the ask as a buy and at or below the bid as a sell, using the tick test inside the spread. The tick test compares the price with the asset's previous print and carries the last non-zero sign through zero ticks. Bulk volume classification assigns each five-second bar a buy share from its last-price change scaled by a twenty-bar rolling standard deviation, then assigns that majority sign to the prints in the bar.

A key design choice is that signs are constructed on every print in each cached daily tape before joining to settlement truth, so unmatched prints stay in the tick history and the five-second bars. Only hash-matched prints can be scored. The paper separates this implementable feed-based history from a settlement-filtered construction that signs only matched prints, and reports both.

The primary ranking uses the intersection signed by all four rules, so every rule faces identical truth observations and class composition. That intersection contains 16,615,817 prints, covering 57.7% to 72.2% of matched prints by day. On that common sample, equal-day balanced accuracy is 0.942 for Lee-Ready, 0.768 for the tick test and 0.670 for the retrospective bulk volume classification diagnostic. Lee-Ready leads on every sampled date, but its score ranges from 0.901 to 0.959.

Accuracy alone does not rank the rules

The paper's central methodological point is that a classification score is not sufficient evidence about the reliability of a trading-cost measure. A wrong sign changes 2s(p-m) by -4s(p-m), so errors on fills where the trade price lies far from the midpoint have larger consequences than errors near it. What matters is the error-weighted signed outcome, not the error frequency.

The own-sample scores illustrate how eligibility changes the picture. Lee-Ready's own balanced accuracy is 0.954 with 100.0% coverage, the quote rule's is 0.953 with 99.8% coverage, the tick test's is 0.735 with 96.0% coverage, and bulk volume classification's is 0.670 with 67.8% coverage. On the common sample the tick test's balanced accuracy rises to 0.768, which the paper says shows that a scoring-population effect should not be mistaken for a construction improvement.

The paper also reports the Matthews correlation beside balanced accuracy, because when one side dominates, raw accuracy is a misleading score. On the common sample the Matthews correlation is 0.809 for Lee-Ready, 0.799 for the quote rule, 0.409 for the tick test and 0.260 for bulk volume classification.

Sign errors move effective spread upward

The economic comparison uses three accounting quantities on identical eligible fills, all in cents of collateral per share and all full-spread quantities that exclude fees, rebates and inventory costs. Effective spread is 2a(p-m), realised spread is 2a(p-m'), and five-minute midpoint impact is 2a(m'-m), where p is the print price, m the preceding delivered midpoint, m' the cached midpoint five minutes later, and a the sign. The three satisfy effective spread equals realised spread plus midpoint impact on every observation.

On 16.4 million identical eligible fills, Lee-Ready raises effective spread by 0.429 cents per share under equal-fill weights, while its realised-spread difference is -0.209 cents. The paper explains the asymmetry: for valid uncrossed current quotes, off-midpoint Lee-Ready errors are wrong-side fills and increase effective spread mechanically, but no analogous direction of distortion follows for realised spread or midpoint impact.

The realised-spread result also shows why signed differences cannot be replaced by absolute error. Positive and negative per-fill differences mostly cancel: the average absolute paired distance is 6.786 cents, although the signed difference between means is only -0.209 cents. The paper states that the effective-spread result does not imply a similar realised-spread or midpoint-impact distortion under another weighting.

Weighting and tied prints flip the impact sign

Under share-volume weights, the taker's five-minute midpoint impact is 0.689 cents, while the tick test and bulk volume classification give -0.374 and -0.119 cents. That aggregate sign reversal is a weighting-dependent result, and the paper tests how fragile it is.

Removing tied scored prints retains 9.1 million observations and changes the aggregate finding: taker, tick and bulk volume-weighted impact are all positive, at 0.955, 0.102 and 0.181 cents. Daily tick and bulk reversals fall to five and two. The paper states plainly that the primary aggregate sign reversal is sensitive to this selection, and that the check retains tied observations in the signing history without recovering venue order or establishing which scoring population is preferable.

Other sensitivities are reported alongside. Restricting the current quote to an age of at most one second retains negative tick and bulk aggregate impact and gives nine and seven daily opposite signs, but Lee-Ready then reverses one daily mean. Substituting the on-chain price leaves impact unchanged by construction and changes Lee-Ready's equal-fill effective distortion only from 0.429 to 0.425 cents.

A single fill shows how a sign error adds four cents

The paper documents one observed signing error in detail. On 17 July 2026 at 15:36:43.018 UTC, a cached print reports a buy of 115.07 shares at 0.22. Its own settlement confirms the same buy, token, price and size. The preceding delivered bid and ask, received 160 ms earlier, are 0.22 and 0.24, giving a midpoint of 0.23. Lee-Ready therefore signs the print as a sell.

Effective spread is -2 cents per share under the taker sign and +2 cents under Lee-Ready, so the sign error adds 4 cents on this fill. The paper is careful about what this does and does not show: the negative taker-signed distance is relative to the delivered quote, and it does not establish the execution-time venue book or a profitable trade.

This example is the mechanism behind the aggregate result rather than an isolated anecdote. It shows why errors concentrated where the trade price lies far from the midpoint can move a signed cost measure even when the overall classification score is high.

Delivered-quote accounting is not causal impact

The paper repeatedly bounds its own economic interpretation. Receipt-time ordering does not identify the execution-time venue book, and a five-minute signed midpoint change is not an identified causal price impact or a measure of private information. Realised spread here is not a fee-adjusted profitability estimate.

The quote used is the asset's last price change strictly before the print's receipt millisecond. Quotes stamped at that same millisecond are excluded because their order relative to the print is unresolved. Forward quote age and future bid and ask are unobserved, so a finite cached midpoint and a within-day horizon do not validate the future book.

The sample itself is twelve selected days chosen to span the archive and venue changes, not a probability sample of trading days. Contracts of the same market or event are dependent, and the small number of days limits inference beyond the observed sample. The paper describes its results as a replicable measurement study of one venue and collector, with explicit boundaries on what the truth labels and quotes observe.

Trust the side field, recompute the signed costs

The practical split the paper supports is between validating an existing field and reconstructing a missing one. The exact side and token agreement on matched settlements establishes what the feed reports: the print corresponds to the settling taker order. Researchers retaining this side field can construct signed measures directly. An archive without trade prices cannot implement the price-based rules benchmarked here, and an archive without the side field must infer direction.

For anyone comparing direction rules, the paper's own ordering is that Lee-Ready leads on balanced accuracy on the common sample, but that ranking does not transfer into a uniform cost distortion. Lee-Ready's equal-fill effective-spread distortion is entirely upward, its realised-spread difference is small and signed negative, and the tick and bulk impact results depend on whether tied print rows are included.

The most decision-relevant threshold in the paper is therefore not an accuracy cutoff but the choice of signed outcome, sample and weighting. Distortion depends on error-weighted signed outcomes, sample selection and weighting, and the paper's own sensitivity checks show that the aggregate tick and bulk sign reversal disappears once tied print rows are excluded. Readers weighing these rules should treat the effective-spread, realised-spread and midpoint-impact comparisons as three separate questions rather than one reliability ranking.

Sources used in this answer