Sensing the Stakeholder: Real-Time Emotion Recognition in Requirements Elicitation
Using Machine Learning to Convey Emotions During Requirements Elicitation Interviews
This paper proposes a real-time emotion detection framework using supervised machine learning to assist analysts during requirements elicitation interviews. By integrating voice recordings and biofeedback data from the Empatica E4 wristband, the system aims to convey the emotional range of stakeholders to reduce ambiguity and communication pitfalls.
TL;DR
Communication is the most fragile part of software engineering. This paper introduces a machine learning framework that uses biofeedback data (from the Empatica E4 wristband) and voice recordings to track a stakeholder’s emotional state during interviews. By detecting discomfort or excitement in real-time, requirements analysts can adjust their questioning strategies to capture more accurate and less ambiguous requirements.
The "Hidden" Signals in Requirements Engineering
In a typical requirements elicitation interview, what a stakeholder says isn't always what they mean. Ambiguity, social pressure, or sensitive topics can lead to "tainted" requirements. Prior work has largely relied on the analyst's intuition to navigate these waters.
The authors' core Insight is that physiological responses—heart rate variability, skin conductance (sweat), and physical fidgeting—provide a "secondary channel" of truth. If an analyst can see a stakeholder's emotional range spiking during a specific question, they can immediately clarify the statement or pivot to a safer topic, preventing project failure down the line.
Methodology: From Biometrics to Emotion
The proposed system doesn't just look at raw numbers; it looks at the shape of human biology.
1. Multimodal Data Fusion
The framework utilizes the Empatica E4 wristband to capture:
- BVP (Blood Volume Pulse): To capture heart activity.
- EDA (Electrodermal Activity): To monitor the sympathetic nervous system (stress/arousal).
- ACC (3-axis Accelerometer): To detect physical fidgeting or movement.
- TEMP: Skin temperature changes.
2. The ANN Architecture
Instead of simple averaging, the authors propose a Polynomial Regression on the BVP signal to capture the actual shape of the stakeholder's pulse. This, combined with voice features, is fed into an Artificial Neural Network (ANN).
The output is a k-bit vector representing three emotional dimensions:
- Valence: Very Negative [0] to Very Positive [100].
- Arousal: Calm [0] to Excited [100].
- Stimulation: Relaxed [0] to Stimulated [100].
Experimental Setup & Data Cleaning
The researchers conducted interviews with 32 participants, who were shown 35 evocative images to calibrate their emotional responses. A critical technical challenge was time-synchronization—matching the Unix timestamps of the physiological data to the moments the participants reacted to specific stimuli.
The team developed custom Python scripts to convert PDF surveys into clean CSV data, using linear interpolation to fill missing values based on stakeholder trends.
Critical Insight & Future Outlook
The real value of this work is not just in "reading minds," but in improving inference speed for real-time applications. By combining audio with biofeedback, the system provides a more robust check than audio alone, which can be easily masked by a stakeholder's professional demeanor.
Limitations:
- Baseline Calibration: Individual "resting" heart rates vary wildly; the model must be highly adaptive.
- Invasive Nature: Wearing sensors during an interview might ironically increase the stakeholder's stress, creating a feedback loop.
Conclusion: This research moves us closer to a future where software requirements aren't just "written"—they are "sensed." By leveraging ANN-based classification of biometric signals, we can turn the subjective art of interviewing into a data-driven science.