Structuring Heritage: Combining PSCAN and LDA for Cultural Discovery in Social Streams

TL;DR

This study presents a scalable pipeline for identifying communities of interest on Twitter regarding cultural heritage. By leveraging Apache Spark and combining PSCAN (structural clustering) with LDA (topic modeling), the researchers successfully mapped user relationships to specific cultural "aspects," achieving a 70% precision rate in real-world Greek heritage datasets.

Context & Motivation: The Digital Heritage Challenge

Social media is a living archive of cultural perception. However, extracting meaningful insights from Twitter streams is notoriously difficult due to:

1. High Volatility: The ephemeral and high-velocity nature of tweets.
2. Sparsity: Short-text documents make traditional NLP difficult.
3. Complex Topology: Social graphs consist of millions of edges where "friendship" doesn't always equal "shared interest."

The authors argue that to truly manage cultural heritage assets, we must find not just who is talking, but what specific historical or artistic niches they are clustering around.

Methodology: The Structural-Thematic Hybrid

The architecture is divided into two major phases: structural identification and thematic extraction.

1. The Power of PSCAN

Instead of simple community detection, the authors use PSCAN (Parallel Structural Clustering Algorithm for Networks). Unlike global partitioning, PSCAN calculates a similarity metric between adjacent nodes: $sim(v,u)=\frac{|N(v)\cap N(u)|}{|N(v)||N(u)|}$ This allows the system to prune weak connections and identify dense "cores" of users who share significant overlap in their social circles.

2. LDA for Aspect Mining

Once communities are identified, the text within each group is processed using Latent Dirichlet Allocation (LDA). By treating each community's collective tweets as a "bag-of-topics," the system identifies the most probable keywords (aspects) that define that specific community.

Implementation & Experimental Results

The system was tested on a dataset of 5,000 tweets focused on Greek heritage (e.g., Corfiot music, ancient paths). The infrastructure utilized Manjaro Linux, Apache Spark 2.2.1, and MongoDB for flexible data storage.

Performance Evaluation

The authors employed a human-in-the-loop evaluation strategy using 10 expert students. The results indicate that the "structural + thematic" approach is highly effective:

This high level of accuracy suggests that users who are structurally connected in "dense communities" on Twitter are indeed discussing highly coherent cultural topics.

Critical Insights & Future Outlook

This work demonstrates that Community Detection is better when it's "aspect-aware." By filtering users through a structural lens first, the noise usually associated with LDA on short-text tweets is significantly reduced.

Limitations:

• The dataset size (5,000 tweets) is relatively small for a "Big Data" framework.
• The reliance on English-only tweets might miss significant local nuances in Greek heritage discussions.

Future Directions: The logical next step is Cross-Language Opinion Mining. As cultural heritage is a global asset, being able to cluster communities across different languages using multilingual embeddings would be a game-changer for international heritage organizations.

Conclusion

By combining the precision of graph theory (PSCAN) with the semantic depth of NLP (LDA), the researchers have provided a blueprint for real-time cultural analytics. It’s a significant step toward turning chaotic social media streams into structured, actionable intelligence for historians and tourism managers alike.