Decoding Professional DNA: CNNs vs. Search-Based Retrieval for Psychological Profiling
Comparison of Convolutional Neural Networks and Search Based Approaches for Extracting Psychological Characteristics from Job Description
The paper introduces and compares two machine learning frameworks—a traditional Search-Based approach (TF-IDF + WMD) and a Deep Learning approach (CNN)—to extract psychological characteristics from job descriptions using the Holland Occupational Themes (RIASEC) model. It aims to automate HR talent matching by quantifying professional strengths directly from textual recruitment data.
TL;DR
Recruitment is no longer just about matching keywords; it is about matching personalities to work cultures. This paper evaluates two distinct paths for extracting Holland Codes (RIASEC) from job descriptions: a Search-Based approach leveraging TF-IDF and Word Mover’s Distance (WMD), and a Convolutional Neural Network (CNN). While the search-based method leads in accuracy (MAE 4.7), the CNN offers a 37.5% win rate in specific traits and superior operational speed.
Problem & Motivation: Beyond Keyword Matching
Traditional HR automation often fails because it ignores the "psychological fitness" of a candidate for a role. Sorting thousands of resumes is a resource-heavy routine. While some researchers use the Big Five (OCEAN) model, this paper argues that the RIASEC (Realistic, Investigative, Artistic, Social, Enterprising, Conventional) model is more effective for identifying professional strengths.
The technical challenge lies in the semantic gap: a job description for a "Designer" might not explicitly mention "Artistic," yet the psychological requirement is inherent. How do we extract these latent traits accurately and efficiently?
Methodology: Two Technical Paths
1. The Search-Based Pipeline (TF-IDF + WMD)
This approach treats the problem as an information retrieval task against the O*NET database.
- Statistical Weighting: Uses TF-IDF to represent long job descriptions as vectors.
- Semantic Precision: Employs Word Mover’s Distance (WMD). Unlike simple cosine similarity, WMD measures the minimum "travel cost" to transform one document's word embeddings into another, capturing deep semantic nuances in job titles.
- Logic: Find the Top-5 most similar professions in O*NET and weigh their pre-defined Holland Codes based on similarity distance.
2. The Deep Learning Pipeline (CNN)
The experts built a regression-based CNN to skip the database lookup:
- Architecture: Words are transformed into word2vec embeddings, passed through a dropout layer, followed by three 1D Convolutional layers.
- Pattern Recognition: CNN kernels act as n-gram detectors, picking up phrase patterns that correlate with specific psychological traits regardless of their position in the text.
Figure 1: The CNN regression architecture for Holland Code prediction.
Experiments & Results: Accuracy vs. Interpretability
The models were tested on manually labeled data from Indeed.com using Mean Absolute Error (MAE).
| Holland Code | MAE (TF-IDF + WMD) | MAE (CNN) |
|---|---|---|
| Realistic | 8.93 | 11.08 |
| Social | 10.60 | 4.39 |
| Enterprising | 12.94 | 9.41 |
| Overall Avg MAE | 4.7 | 5.9 |
Key Insights:
- Search-Based Wins on Average: Because it references the curated O*NET database, the search-based model is more "grounded." It is also highly interpretable—HR managers can see which professions the model matched with the vacancy.
- CNN Wins on Speed and Specificity: The CNN significantly outperformed the search-based model in the "Social" and "Enterprising" categories. Furthermore, once trained, the CNN does not require expensive database similarity calculations, making it ideal for high-throughput software systems.
- Visualization:
Figure 2: Predicted Holland Codes for a "Designer" query, showing the distribution across RIASEC dimensions.
Critical Analysis & Conclusion
Takeaway
There is no "one size fits all" algorithm here. If your priority is interpretability and reliability, the TF-IDF + WMD approach is superior as it acts as a smart "lookup" tool. If your priority is inference speed and capturing complex patterns in specific traits like social skills, the CNN is the better choice.
Limitations
- The Black Box Problem: The CNN remains difficult to interpret. In HR, knowing why a candidate is classified a certain way is legally and ethically vital.
- Data Sparsity: The CNN was trained on approximately 1,100 job descriptions; larger datasets would likely push the CNN's performance past the search-based baseline.
Future Work
The authors suggest moving the entire stack to the Julia language using the Flux library to capitalize on high-performance computing gains. Moving forward, integrating these models into a unified voting system could leverage the precision of search with the speed of neural networks.