[JCDL 2020] Gandhipedia: Mapping a Prophet’s Life through AI and Network Science

Executive Summary

TL;DR: Gandhipedia is a sophisticated AI-driven portal that transforms the vast, unstructured literary legacy of Mahatma Gandhi into an interactive web experience. By leveraging ensemble NLP techniques and network science, the platform reconstructs Gandhi's life as a dynamic social network, allowing users to explore his connections to people and places across time and space without manual reading.

Context: This work is a hallmark of Digital Humanities. Positioned between traditional archival science and modern Information Retrieval, it moves beyond simple digitization (PDFs) to actionable knowledge representation, setting a SOTA benchmark for historical figure portals.

Problem & Motivation: The Fatigue of Information Overload

Historical archives for figures like Gandhi are massive—reaching over 100 volumes of speeches, letters, and books. For a researcher or a curious citizen, finding specific mentions of a person (e.g., Gopal Krishna Gokhale) across decades of text is like finding a needle in a haystack.

The Pain Point: Existing platforms provide access but not insight. They offer search boxes but fail to show the relational context—how Gandhi’s social circle evolved or how his presence in specific cities changed over time. The authors identified that users disengage quickly when faced with static text; they need a visual, interactive entry point.

Methodology: The Architecture of Digital Biography

The authors don't just rely on one tool; they build a pipeline that ensures data integrity and high-quality extraction.

1. The Data Pipeline

The core data source is the Collected Works of Mahatma Gandhi (CWMG). The system converts raw PDFs into XML to preserve structure, specifically identifying chapter boundaries which serve as the temporal unit of analysis.

2. Ensemble NER (The Secret Sauce)

Entity extraction in historical texts is notoriously noisy. To combat this, the researchers used an Ensemble Strategy:

• Tools: NLTK, Polyglot, and SpaCy.
• Logic: A "Majority Voting" mechanism—an entity is only accepted if at least two of the three libraries identify it.
• Refinement: WordNet is used to filter out common nouns, ensuring nodes are strictly people or places.

3. Temporal Social Networks

This is the most innovative part of the methodology. Instead of a static graph, they mapped entities to specific years.

• Edges: Two entities are connected if they appear in the same year.
• Temporal Smoothing: To account for ongoing relationships, entities in year t are also linked to those in t-1 and t+1.
• Community Detection: Algorithms like Louvain and Infomap are applied to color-code clusters (e.g., social groups or regional associates).

Experiments & Results: Navigating History at a Click

The portal provides two primary ways to interact with history:

1. Network-Driven Exploration: Users can view the "People Graph" or "Place Graph." Clicking a node (like a specific person) triggers a backend query that pulls all excerpts where Gandhi mentioned that individual, organized chronologically.
2. Full-Text Search: Using Elasticsearch, the portal allows for rapid keyword searches across 40 volumes (expanding to 100), highlighting results within the original text context.

The visualization above shows how entities are clustered; for instance, identifying the "South Africa" phase vs. the "India Independence" phase of Gandhi's life based on entity co-occurrences.

Critical Analysis & Conclusion

Takeaway

Gandhipedia successfully bridges the gap between text mining and public history. It proves that structural visualization is the future of digital libraries. By treating a person's life as a dataset of relationships, we can gain "bird's eye" insights that are impossible through linear reading.

Limitations & Future Work

• Multilingual Support: Much of Gandhi's influence and writing occurred in local Indian languages; the authors plan to expand to multilingual search.
• First-Appearance Timelines: A planned feature is a chronological timeline marking the "first mention" of significant figures, which would provide even better narrative structure.
• Refinement: While ensemble NER is robust, historical slang or archaic place names might still require manual fine-tuning or specialized LLM (Large Language Model) intervention in future iterations.

Final Verdict: Gandhipedia is a pioneering platform that sets the standard for how we should preserve and interact with the legacies of world leaders in the 21st century.