EnHMM: Predictive Bug Triaging through the Lens of Sequential Stack Traces

Executive Summary

TL;DR: Bug reports are the lifeblood of software maintenance, yet they are often riddled with incorrect metadata. EnHMM is a sophisticated ensemble framework that treats bug triaging as a sequence modeling problem. By training Hidden Markov Models (HMMs) on the function call sequences within stack traces, this approach achieves a massive 36% gain in recall over previous benchmarks, fundamentally reducing the "bug-pong" phenomenon where reports are tossed between teams due to mislabeling.

In the landscape of software reliability engineering, this work represents a transition from "What the user said" (NLP) to "What the system actually did" (Stack Trace sequence analysis).

The "Bug-Pong" Problem: Why Descriptions Aren't Enough

When a user submits a bug, they often guess the "Component" or "Priority." Statistical evidence shows that up to 80% of bug reports require field reassignment. Previous SOTA methods, such as Im.ML.KNN, focused heavily on the natural language summary and description.

The Insight: Natural language is ambiguous and prone to human error. Conversely, a stack trace is a deterministic footprint of the crash. However, traditional ML models treat stack traces as "bags of words," losing the vital temporal order of function calls. EnHMM views these traces as stochastic processes where the sequence of calls reveals the true nature of the fault.

Methodology: Harnessing the Power of Ensembles

The core of EnHMM lies in its multi-layered training and selection pipeline:

1. Sequential Extraction: Using optimized regular expressions, the system pulls function call sequences from raw Bugzilla data.
2. HMM Training: Two distinct sets of HMMs are trained—one for the rare class (Reassigned) and one for the majority class (Not-Reassigned). By varying the hidden states ($N=\mathrm{10...200}$), the authors capture different levels of abstraction in the execution flow.
3. Diversity-Driven Selection (WPIBC): Instead of a simple average, EnHMM uses Weighted Pruned Iterative Boolean Combination. It calculates Cohen’s Kappa to identify and remove redundant HMMs, keeping only the most "diverse" detectors.

Fig 1: The EnHMM Workflow - from data splitting to Boolean combination rules.

Experimental Showdown: Superior Recall

The authors tested EnHMM against the Eclipse and Gnome repositories. The results highlight a critical trade-off in industrial bug triaging: Precision vs. Recall.

• The Single HMM Gap: EnHMM showed a 76% improvement over the best single HMM in the Gnome dataset, proving that a single HMM is too "brittle" for varied execution paths.
• Beating the SOTA: Compared to the text-based Im.ML.KNN, EnHMM improved the F-measure for the "Priority" field by 17.35% and the "Status" field by a staggering 147.25%.

Fig 2: ROC Curves for Eclipse fields. The red line (EnHMM) consistently maintains the highest AUC.

Critical Analysis: The Precision Trade-off

While the recall is impressively high (76.39%), the precision (53.93%) is slightly lower than text-based models.

• The "Why": Stack traces are pure technical signals. They are excellent at identifying if a bug is complex (needs reassignment), but without the context of the user's intent (the text), they can occasionally flag reassignments where none are needed.
• Scalability: A standout feature of this research is the Kappa-based pruning. The system reduces 40 candidate detectors down to just 6-8, making it efficient enough for real-time triaging in CI/CD pipelines.

Future Outlook: The Hybrid Path

EnHMM proves that sequential code data is a goldmine for maintenance prediction. However, the future likely belongs to Hybrid Models. By combining the sequential HMM logic for stack traces with Transformer-based models (like BERT) for text descriptions, we can bridge the gap between human context and machine execution.

Key Takeaways for Engineers:

• Trust Traces: If your bug tracking system doesn't enforce stack trace attachments, you are losing 90% of your predictive power.
• Ensemble is Essential: When dealing with imbalanced bug data, combining "diverse" weak learners is more robust than fine-tuning a single giant model.