Mining Deception: Unveiling the Patterns of Political Truthfulness in Serbia

TL;DR

In an era of political transition and misinformation, how can we hold officials accountable? This paper presents a data-mining pipeline that analyzes the "Truth-o-meter" (Istinomer) registry in Serbia. Using character n-grams and Hubness-based analysis, the study uncovers that Serbian politicians are significantly less truthful than those in Western democracies—with an average fulfillment rate of just 30%—and provides a framework to predict whether a promise is likely to be kept.

Background: Data Science as a Democratic Shield

For countries like Serbia, historical corruption and political shifts have often led to voter cynicism. While systems like PolitiFact (US) and Istinomer (Serbia) collect expert-annotated data, the data's volume makes it difficult for humans to spot systemic trends. This research treats political promises not just as text, but as high-dimensional data points that can be mined for "pathological" patterns of deceit.

Methodology: High-Dimensional Insights

The author faces a unique challenge: the Serbian language lacks the robust NLP toolkits available for English. To solve this, the study adopts a language-independent approach.

1. Vector Space & N-Grams

The system uses 4-character n-grams instead of words. This avoids the need for complex Serbian stemming while capturing morphological nuances. Textual similarity is then calculated using high-dimensional cosine similarity.

2. Hubness as Centrality

A key innovation here is the use of Hubness. In high-dimensional spaces, certain points (hubs) appear as the "nearest neighbors" to an unusually large number of other points. The author leverages these hubs to identify "Typical Promises"—prototypes that summarize the core themes of political discourse.

Figure 1: The proposed multi-step pipeline for truthfulness pattern detection.

Key Discoveries: The Anatomy of a Lie

The analysis reveals several disturbing and fascinating trends:

• The Truthfulness Deficit: The average truthfulness score is 0.305, significantly lower than the ~67% reported in some Western European studies.
• Topic-Specific Deceit: Association rules (e.g., Rule AR4: Elections + European Union ⇒ Unfulfilled) show that promises involving infrastructure and EU integration during election cycles are almost never met.
• Political Similarity: By averaging promise profiles, the system automatically correctly clustered politicians by their roles (e.g., mapping ministers of agriculture or energy together), proving that the n-gram model captured deep semantic roles.

Figure 2: Average truthfulness by topic. Note the low scores for infrastructure and EU-related promises.

Experiments & Prediction Results

Can we predict if a politician is lying in real-time? The author tested several models, including HIKNN (Hubness-Information k-Nearest Neighbor) and Naive Bayes.

• Performance: While the "Zero-rule" (predicting everyone is lying) had high accuracy due to the dataset's skew, content-based models achieved superior Macro-F1 scores, meaning they actually attempted to distinguish between types of promises.
• Predictability: Some politicians are "predictably untruthful," while others are more erratic. The system was best at estimating truthfulness regarding Kosovo and Privatization topics.

Table 1: Comparative performance of various truthfulness estimators.

Critical Analysis & Conclusion

The study’s greatest strength is its modality-agnostic nature. It doesn't need to "understand" Serbian grammar to find truthfulness patterns; it finds them through the geometry of high-dimensional space.

Limitations:

1. Data Bias: The Istinomer team might selectively track more prominent or controversial figures.
2. Fuzzy Labels: Quantifying a qualitative label like "almost fulfilled" with a value of 0.75 is inherently subjective.

Future Work: The author suggests moving toward Representation Learning (e.g., early Word2Vec models like Mikolov et al. 2013) and automated promise extraction from news corpora to scale this "Democratic Information System" globally.

By bridging the gap between machine learning and political science, this work provides a blueprint for a future where policy-makers are held to a standard that is both transparent and mathematically verifiable.