[IP&M] Beyond TF-IDF: Boosting Retrieval Precision via Document Context Similarity

TL;DR

Researchers from the University of Tehran have developed a novel query term re-weighting mechanism that identifies "informative keywords" by looking at how top-retrieved documents relate to one another. By emphasizing terms found in the "consensus context" of the initial results, the method achieves a ~7% MAP boost across multiple languages, proving particularly effective for both short and verbose queries.

The "Query Drift" Dilemma

In the world of Information Retrieval (IR), Pseudo-Relevance Feedback (PRF) is a double-edged sword. It assumes that the first few documents retrieved for a query are relevant and uses them to refine the search. However, if the query is vague (e.g., "Frank Sinatra in L.A."), the top results might be a mix of concert reviews, biographies, and news snippets.

Traditional methods often struggle here:

1. Query Expansion might add noisy, irrelevant terms.
2. Simple Re-weighting might over-emphasize a term just because it appears frequently in a single, non-relevant top document.

The authors' insight? The most "truthful" query context lies in the similarity between the top documents. If multiple top documents share a similar language model, they likely represent the user's actual information need.

Methodology: The DS Weighting Framework

The core of the proposed method is a refined weighting formula that moves beyond simple term frequency.

1. Document Relevance ($v_{d_j}$)

Instead of trusting all top $N$ documents equally, the system weights each document $d_j$ based on its average similarity to all other documents in the set. This acts as a density-based filter—documents in the "center" of the result cluster get more vote in re-weighting query terms.

2. The $Q_i$ Innovation (Anti-Bias)

To prevent a single query term $q_i$ from artificially inflating a document's relevance, the authors calculate document-to-query similarity using $Q-\{q_i\}$. This ensures that a term $q_i$ only gets a high weight if its containing documents are also relevant to the rest of the query.

The final weighting equation (Eq. 10) integrates Term Frequency, IDF, document-set similarity, and the leave-one-out query similarity.

Experimental Results & Robustness

The model was tested on the FIRE (English) and Hamshahri (Persian) datasets. The results consistently showed that the "DS Weighting" outperformed both standard Language Modeling (LM) and the Weighted Information Gain (WIG) method.

Table 3: Significant MAP gains observed in Persian datasets, specifically highlighting the robustness of the method.

Key Findings:

• Stability: Unlike many PRF methods that degrade as more documents ($N$) are added (bringing in noise), the DS method remained stable even at $N=300$.
• Short Query Success: Even for 3-word queries, identifying the one "key" term led to performance jumps of over 50% in specific cases (e.g., the query for "Sadi Shirazi").
• Efficiency: While requiring two retrieval passes, it is faster than expansion-based methods because the "dictionary" size of the query remains small.

Critical Insight & Future Outlook

This work demonstrates that "Internal Consensus" is a powerful signal in IR. While modern LLM-based rankers are currently in favor, this algebraic re-weighting approach offers a computationally cheap and transparent way to improve search engines without the "black box" nature of neural networks.

Limitations: The method relies on the "Cluster Hypothesis." If the initial results contain zero relevant documents (the "Hard Query" problem), the method can still suffer from drift, as seen in the "Mental Health" query example in the paper.

Conclusion: For developers and researchers building search systems for non-English languages or specialized corpora, the DS weighting approach provides a robust, easy-to-implement strategy for squeezing more precision out of the original user query.