mLTL: Mastering the Sequence of Knowledge for Deep Learning Breakthroughs

TL;DR

The performance of a Deep Neural Network (DNN) isn't just about the amount of data, but the order in which it learns. This paper proposes Multilevel Transfer Learning (mLTL), a framework that sequences multiple related datasets to boost performance in Facial Emotion Recognition (FER) and Named Entity Recognition (NER). By moving beyond simple one-to-one transfer, mLTL achieved up to a 16.3% accuracy boost in facial expression tasks and brought lightweight models like DistilBERT close to heavy SOTA performance.

Problem & Motivation: The "Single-Jump" Limitation

Standard transfer learning usually follows a simple path: Pre-trained Model (e.g., ImageNet/BERT) $\to$ Target Task.

However, the authors identify a critical bottleneck: if the target dataset is small or the task is highly specialized (like recognizing "E-sports coaching emotions"), the jump from general knowledge to specific application is too wide. This often leads to:

1. Overfitting: The model memorizes noise in the small target set.
2. Feature Inefficiency: General features aren't properly "morphed" into task-specific features.

The authors' intuition is that intermediate steps act as a ladder, gradually refining the model's internal representations.

Methodology: The mLTL Framework

The core of mLTL is the identification of $N$ datasets related to the task and organizing them into $m$ levels of transfer.

1. Computer Vision: Dense_FaceLiveNet

For FER, the authors evolved the architecture to be more robust for transfer:

• Dense Blocks: Replacing Residual blocks to ensure maximum feature reuse.
• Global Average Pooling (GAP): Replacing heavy Fully Connected layers to reduce parameters and combat overfitting.
• Swish Activation: Using a smooth, non-monotonic function to improve gradient flow compared to ReLU.

2. Natural Language Processing: DistilBERT

For NER, the framework leverages DistilBERT, a model that is 40% smaller and 60% faster than BERT, proving that mLTL can make efficient models compete with larger ones.

3. The Four Golden Principles of Sequencing

Perhaps the most valuable contribution of the paper is the discovery of how to order datasets:

• Data Volume Matters: Larger datasets should generally be used later in the sequence to prevent "forgetting" once features are refined.
• Similarity Proximity: Datasets with similar label counts or granularities should be placed closer together.
• The Specificity Rule: When datasets are dissimilar, place the larger one at the beginning of the sequence to provide a broad feature base.

Experiments & Results: Quantitative Triumphs

The authors tested mLTL across diverse datasets (e.g., CK+, FER2013, MSRA, Weibo NER).

Facial Emotion Recognition

Before mLTL, the GFE2019 dataset accuracy was a mere 71.53%. By applying a 3-level transfer (CK+ $\to$ FER2013 $\to$ LE $\to$ GFE2019), accuracy surged to 87.84%.

Named Entity Recognition

The results for Chinese NER were equally impressive. DistilBERT, which typically struggles on the specialized "Resume" dataset (F1: 65.42%), reached 93.49% using the $O\to M\to W\to R$ sequence.

Efficiency Highlight: While models like Glyce+BERT achieve slightly higher F1-scores (96.54%), DistilBERT via mLTL is 3x faster, making it viable for real-time edge applications.

Critical Analysis & Conclusion

Takeaway

mLTL proves that knowledge distillation and transfer are not mutually exclusive but additive. By carefully curating the "educational path" of a model, we can achieve high performance with significantly lower computational costs.

Limitations

• Sequence Search: Finding the optimal sequence among $T_m^n$ combinations (as defined in the paper) can be computationally expensive as $n$ increases.
• Catastrophic Forgetting: While mLTL improves the target task, the paper does not extensively measure how much knowledge of the original source domains is retained.

Future Outlook

The principles of mLTL could be automated using Neural Architecture Search (NAS) or RL-based controllers to find the optimal training sequence, potentially revolutionizing how we handle "Small Data" problems in industry.