Engineering Insights: Navigating the Hierarchy of Scientific Literature Summarization

TL;DR

In the world of Academic NLP, understanding why a paper is cited is as crucial as knowing if it is cited. This paper details a winning entry for the CL-SciSumm 2016 challenge, introducing a robust pipeline that links citations to specific source text, classifies them into functional facets (Method, Aim, etc.), and generates structured summaries using hierarchical topic modeling (hLDA).

The "Detached Citation" Problem

Most citation analysis treats a reference as a simple pointer. However, the authors argue that citations are currently "detached" from the source. When a scholar cites a work, they might be referencing a specific result, a shared hypothesis, or a specific method. Without automated Citation Linkage, we lose the granular context of scientific evolution. Furthermore, generic summarizers often ignore these links, resulting in abstracts that fail to highlight what the broader scientific community actually finds valuable in a paper.

Methodology: Beyond Simple Keywords

The authors break the problem into three sequential technical challenges:

1. Citation Linkage (Finding the Source)

To find exactly which sentences in a Reference Paper (RP) correspond to a citation in a Citing Paper (CP), the team used an ensemble of six similarity measures.

• Lexicons: Building high-frequency word lists and expanding them via WordNet.
• Semantic Vectors: Using Word2Vec and Doc2Vec to calculate cosine similarities between sentences, allowing the system to find linkages even when exact keywords don't match.
• The "Jaccard Focused" Method: Interestingly, the authors found that the simple ratio of shared words (Jaccard) often outperformed complex vectors on small academic datasets.

2. Facet Classification: The Functional Context

Once linked, the text is categorized into one of five facets: Aim, Method, Results, Implication, or Hypothesis.

• The Voting Method: They combined a rule-based subtitle analysis (e.g., searching for "Experimental Results" headers) with an SVM classifier.
• Fusion Strategy: By aggregating results from multiple runs, they achieved a significant lead over other competitors.

3. Summarization with hLDA

The "secret sauce" of their summarization is hierarchical Latent Dirichlet Allocation (hLDA).

• Visualizing Hierarchy: Unlike standard LDA, hLDA organizes topics in a tree. The path from the root to a leaf represents a sentence's "abstractiveness."
• Scoring: Sentences assigned to specific levels of the tree are weighted based on their representativeness of the document's core message.

Figure 1: The solution framework for link recognition and facet classification.

Experimental Performance

The system's performance peaked when combining structural features (like sentence position) with the semantic insights of hLDA.

While Sentence Position remains a dominant heuristic in scientific writing (important info is often at the start or end), the hLDA and spans features provided the necessary nuance to win the "Community Summary" task, which reflects how other scholars view the paper.

Figure 2: Performance comparison across different systems in the CL-SciSumm challenge.

Critical Insight & Conclusion

This work highlights a critical lesson for Technical Editors and Researchers: Data Imbalance is the Enemy. The researchers struggled with SVMs because the number of non-cited sentences (negative samples) was 125 times larger than cited ones (positive samples). Their pivot to a heuristic/voting ensemble is a practical masterclass in handling small-scale, high-specificity academic datasets.

Future Outlook: As we move into the era of LLMs, the hierarchical structures identified by hLDA could serve as powerful "Chain-of-Thought" templates for more advanced abstractive summarizers.