[BISTU 2020] Mining the Dev-Social Graph: SFF-Specific NER using BI-LSTM and Word Embeddings

Executive Summary

TL;DR: This paper tackles the "noise" of software development social networks (like CSDN and Stack Overflow) to extract structured knowledge. By deploying a BI-LSTM network with Word2Vec embeddings, the authors identify critical software function features (SFF), bridging the gap between raw forum discussions and structured knowledge graphs.

Positioning: This work acts as a foundational domain-adaptation study, moving NER from traditional "News/Medical" domains into the specialized, messy world of Software Engineering (SE).

Problem & Motivation: The "Free-Form" Chaos

Software developers communicate in a hybrid language. A single post on CSDN might contain:

• Natural Language: Chinese or English descriptions.
• Technical Identifiers: APIs, Frameworks, and Platforms.
• Noise: Snippets of source code, URLs, and non-standard abbreviations.

Existing SOTA models for news (identifying People/Orgs) fail here because they cannot distinguish between a variable name in code and a critical software entity in text. Furthermore, previous SE-NER models ignored "Undefined Functions"—custom function names that carry heavy semantic weight but are almost always Out-of-Vocabulary (OOV).

Methodology: Bridging Code and Context

The authors propose a three-stage pipeline to handle this complexity:

1. Data Sanitization

To prevent the model from getting lost in actual code logic, they use regular expressions (following Dagenais et al.) to strip out block code and links, focusing the recognition purely on the "Software Function Feature" descriptions.

2. Semantic Vectorization (Word2Vec)

They utilize the Skip-gram model with negative sampling. Since the data contains English terms (Java, Python) inside Chinese sentences, they opted for Word Embedding over Character Embedding to better capture the semantics of technical terms.

3. The BI-LSTM Architecture

The core of the system is a Bidirectional LSTM that processes sequences with the BIOES (Beginning, Internal, End, Other, Single) tagging schema.

Figure 1: The dual-layer BI-LSTM framework designed to condense 128-dimensional vectors into tag probabilities.

The model utilizes a CRF (Conditional Random Field) layer logic (implied via the Viterbi algorithm in the text) to find the highest scoring label sequence, ensuring that the predicted tags (like B-PL followed by I-PL) are logically consistent.

Experiments & Results: The "Undefined" Challenge

The system was tested on a dataset of Python-related Q&A from CSDN.

Key Insight from Ablation/Error Analysis: The model struggled most with the "Undefined Function" (UF) category. Why? Developers use highly inconsistent naming conventions for their own functions (e.g., exchangeTwoNumber() vs printResults()). These appear infrequently, leading to a lack of training samples and a high OOV rate.

Figure 2: Example of manual entity annotation versus model recognition of frames and functions.

Critical Analysis & Conclusion

Takeaway

The inclusion of the "Undefined Function" category is a significant step toward "true" understanding of developer intent. By recognizing these, researchers can eventually map specific user issues to specific logic blocks in a Knowledge Graph.

Limitations

• Temporal Decay: Word2Vec embeddings are static. In the fast-moving software world, new libraries (e.g., Mojo, JAX) would require retraining the entire embedding space.
• Context Window: The sequence length is capped at 80 words, which might truncate complex technical discussions.

Future Outlook

The authors aim to move toward Relationship Extraction. Identifying that "Python" (PL) uses "NumPy" (Fram) is the next logical step in building a comprehensive Software Engineering Knowledge Graph.