Quality Over Quantity: Solving Label Noise in Emotion Recognition via Instance Selection

TL;DR

In the realm of Affective Computing, more data isn't always better. This paper demonstrates that by using Instance Selection, we can throw away up to 98% of noisy training data and actually increase emotion recognition accuracy from 65.9% to over 90%. By focusing on high-confidence "prototypical" emotional segments, the authors solve the mismatch between global experimental labels and local acoustic features.

The Problem: The "Global vs. Local" Labeling Paradox

In naturalistic human-computer interaction (HCI), such as the Wizard-of-Oz (WoZ) experiments described in this study, emotions are not constant. An investigator might label a 5-minute block as "Negative Valence," but within those 5 minutes, the user is only truly showing frustration for a few seconds.

The technical roadblock is clear:

1. Global Labels: Coarse temporal scales (minutes).
2. Local Features: Audio analysis windows (milliseconds).
3. The Noise: Most windows in a "Negative" block are effectively neutral, leading to a dataset filled with contradictory signals that confuse standard classifiers like SVMs.

Methodology: The Confidence-Based Filter

The authors propose a two-stage architecture designed to distill the "essence" of an emotion from a sea of noise.

1. Reclassification via Cross-Validation

Instead of trusting the manual labels blindly, the system "interrogates" the data. The dataset $T$ is split into $N$ bags. An SVM is trained on $N-1$ bags and used to predict the class and confidence of the remaining bag.

2. The Selection Logic

After reclassifying the entire set, the algorithm performs two critical cuts:

• Consistency Check: Remove all instances where the SVM's prediction disagrees with the original (noisy) label.
• Density Reduction: Keep only the top $r$ instances with the highest probabilistic confidence.

Figure 1: The Instance Selection Algorithm pseudocode highlighting the confidence-based filtering.

Experiments: Why 2% is Better Than 100%

The evaluation was conducted on subjects using the "Concentration" game setup, inducing emotions via system delays and incorrect speech recognition.

Performance Gains

The most striking result is the relationship between data reduction and accuracy. When the training set was reduced to just 2% of its original size—retaining only the "purest" emotional samples—the system achieved:

• Accuracy: 96.0% (with a 0.95 confidence rejection threshold).
• F1-Measure: 0.965.

Even without rejecting samples during testing, the pruned dataset yielded a 68.5% accuracy compared to 65.9% for the full dataset, with much lower standard deviation.

Figure 2: Accuracy rises sharply as the confidence threshold increases, proving that the model's self-assessment of "certainty" correlates with ground truth.

Deep Insight: The Value of Rejection

One of the paper's key contributions is the use of the Rejection Option. In real-world HCI, it is often better for a system to say "I don't know" than to guess a user's emotion incorrectly. By setting a confidence threshold of 0.95, the system filters out ambiguous "neutral" audio segments that likely shouldn't have been labeled as emotional in the first place.

Critical Analysis & Conclusion

Takeaway

This research highlights a vital lesson for AI practitioners: in domains where labeling is difficult (Audio, Bio-signals, Medical imaging), automated data cleaning via instance selection is more effective than manual intensive labeling. It effectively acts as a filter for "Inductive Bias," forcing the model to learn from the clearest signals.

Limitations & Future Work

While the results are impressive, the study focuses on a small cohort (6 subjects). The "individuality" of emotion means these filtered signatures might not generalize across a wider population without Transfer Learning. The authors suggest that moving toward multi-modal co-training (combining audio with video or physiology) could further refine the selection process, using one modality to "verify" the labels of another.

Ultimately, this work proves that for robust emotion AI, we must move away from "Big Data" and toward "Smart Data."