Beyond Simple Synonyms: Decoding Finnish Lexical Choice with 650+ Linguistic Features

TL;DR

Choosing between "think," "reflect," and "ponder" is more than a stylistic flip of a coin—it is a complex linguistic puzzle. Researchers from Aalto University and the University of Helsinki have benchmarked how machines handle this via the amph dataset, using over 650 features. The results show that while more data usually helps, too much linguistic detail acts as noise unless filtered through rigorous feature selection, capping supervised accuracy at around 66%.

The "Curse of Detail" in Lexical Choice

In Natural Language Generation (NLG), picking the right word to fill a gap (Fill-In-The-Blank) is a core challenge. Previous work often relied on small, manually curated feature sets. This paper asks a bold question: If we give a model every possible linguistic detail—morphology, syntax, and semantics—can it achieve perfect performance?

The answer is a surprising "No." The authors discovered that an extensive feature set creates a "high-dimensional noise" problem. In Finnish, a language with rich agglutinative morphology, the context of a word like miettiä (reflect) is so complex that unsupervised models (like K-means or SOM) struggle to find the "lexeme signal" amidst the "linguistic noise."

Methodology: High-Dimensional Mapping

The authors employed a diverse arsenal of machine learning techniques to probe the amph dataset (3,404 occurrences of four Finnish "think" verbs):

1. Unsupervised Exploration: Using Self-Organizing Maps (SOM) to visualize the manifold of the data.
2. Automated Feature Selection: A forward heuristic to find the "Goldilocks" zone—the FS40 set (40 features) proved to be more efficient than the full set of 651.
3. Supervised Benchmarking: Comparing Neural Networks (ANN), Multinomial Logistic Regression (MNR), and k-Nearest Neighbors (kNN).

The SOM visualization reveals that lexeme clusters (represented by different gray-scale bars) are heavily overlapped, illustrating why unsupervised learning finds this task nearly impossible.

Key Insights from the Results

The experiments yielded several critical takeaways for computational linguists:

• The Power of Syntax: Of the top 10 features selected by the FS40 algorithm, 9 were syntactic. Pure morphology and extra-linguistic features (like author info) were far less predictive.
• The 66% Ceiling: Regardless of the model (ANN, MNR, or kNN), accuracy consistently plateaued around 60–66%. This suggests a fundamental limit to how much "word choice" is determined by immediate linguistic context versus idiosyncratic human preference.
• Unsupervised Failure: Without labels, models couldn't beat the 44% baseline. This implies that the features which define a "context" in language often reflect topic or style rather than the specific nuances of near-synonyms.

Comparison of accuracies across different models and feature sets (FULL vs. FS40 vs. Atomic).

Critical Analysis & Future Outlook

This work serves as a sobering reminder for AI researchers: more features do not equal better models. The "amph" dataset proves that near-synonymy is a "soft" boundary problem.

Limitations: The reliance on partially manual linguistic analysis makes scaling this to all lexemes in a language difficult. Furthermore, the 66% accuracy ceiling suggests that we might need to look beyond the immediate sentence—perhaps into broader discourse or even the speaker's mental state—to truly solve lexical choice.

Future Work: The next logical step is moving from hand-crafted linguistic features to Deep Contextualized Embeddings (e.g., Transformers). Can a model like Finnish BERT capture the "hidden" signals that the 651 manual features missed?

Conclusion

By systematically breaking down the "think" lexemes of Finland, the authors have provided a roadmap for feature engineering in NLP. They've proven that while syntax is king, the "noise" of language is a formidable foe that only smart feature selection can defeat.