ReqSimile: Bridging the Gap Between Customer Wishes and Product Reality via Linguistic Engineering

TL;DR

Managing thousands of requirements in a market-driven software company is a "needle in a haystack" problem. Researchers from Lund University and the University of Pisa have developed a pragmatic approach using the Vector-Space Model to automatically suggest links between raw customer feedback and internal business requirements. Their tool, ReqSimile, demonstrates that statistical linguistic techniques can reduce manual requirements-tracing effort by over 60%.

The "Requirements Explosion" Problem

In large-scale software production, requirements management isn't just about writing specs; it's about linking. On one side, you have Customer Wishes (Market Requirements)—unstructured, diverse, and often repetitive. On the other, you have Product Requirements (Business Requirements)—structured, technical, and development-ready.

The bottleneck arises because these two worlds emerge independently. As products grow, manual searching for "Is this new customer request already covered by an existing requirement?" becomes financially and cognitively unsustainable. Deteriorating hierarchies and evolving architectures make traditional folder-based organization obsolete.

The Core Intuition: Lexical Similarity as a Proxy for Meaning

The authors acknowledge a hard truth: in the early 2000s (and even today to some extent), perfectly capturing "meaning" via AI is elusive. Instead, they leverage a pragmatic assumption: in a specific technical domain, if two requirements use the same terminology, they likely refer to the same functionality.

The Methodology Pipeline

1. Preprocessing: Flattening text, removing punctuation, and Stemming (e.g., converting "managed" and "managing" to "manage").
2. Vectorization: Transforming each requirement into a high-dimensional vector where each dimension is a unique word count.
3. Log-Weighting: Applying $1+{\log }_2(exttermfrequency)$ to ensure that common words don't disproportionately skew the similarity.
4. Cosine Similarity: Measuring the angle between these vectors to rank potential matches.

Industrial Evidence: The Baan Case Study

The authors tested their approach on a massive real-world dataset from Baan (now part of SSA Global). The scale was significant:

• 8,300 Market Requirements.
• 3,800 Business Requirements.
• Over 4.5 million lines of code.

Key Findings

The researchers compared the automated suggestions against manual links created by product managers.

• Recall Performance: In a "top-10" recommendation list, the system achieved a 51% recall. While not 100%, it provides a high-quality starting point that eliminates the need to search the entire database.
• Efficiency Gain: By narrowing the search space from ~30 results (traditional keyword search) to 10 (automated similarity), the authors estimate a 66% saving in manual effort.

Deep Insight: Why This Works

The value of this research lies in its robustness and scalability. Unlike complex NLP parsers that break on typos or unconventional grammar (common in customer emails), the Vector-Space Model is resilient. A typo in one word only slightly affects one dimension of a vector, leaving the overall similarity score largely intact.

Furthermore, the ReqSimile tool provides a human-in-the-loop interface. It doesn't replace the Product Manager; it acts as a recommendation engine, allowing experts to confirm or reject links with a single click, thereby maintaining the "ground truth" while accelerating the workflow.

Limitations and Future Outlook

The main limitation is "semantic distance"—the model fails if two requirements describe the same feature using entirely different vocabularies (e.g., "UI" vs "Frontend").

The authors suggest that future iterations could incorporate thesauri (synonyms/hypernyms) and machine learning to derive relevance from previously confirmed links. In today's context, this work sets the stage for modern Transformer-based embeddings which can capture these semantic relationships even more deeply.

Conclusion

This paper is a classic illustration of Linguistic Engineering: taking established Information Retrieval (IR) theories and applying them to solve a tangible, high-stakes industrial bottleneck. For any organization dealing with an "inflow" of customer data, the transition from keywords to lexical similarity is the first step toward intelligent requirements management.