Precise and Equitable: Redefining Recidivism Prediction with Granular Criminal History
Reducing race-based bias and increasing recidivism prediction accuracy by using past criminal history details
This paper introduces a neural network-based framework for recidivism prediction that improves accuracy while mitigating racial bias. By utilizing high-resolution criminal history (rolling sums of 26 crime categories) and "Fairness through Unawareness," the authors identify optimal model configurations that achieve state-of-the-art predictive performance on the 1994 US Prisoner Release dataset.
TL;DR
Predicting recidivism—the likelihood of a convict re-offending—is a high-stakes task where algorithmic bias can lead to systemic injustice. This research proves that we don't have to trade accuracy for fairness. By utilizing rolling sums of detailed criminal history (splitting records into 442,000 individual arrest cycles) and a selection strategy based on False Positive Rate (FPR) Parity, the authors achieved a massive jump in accuracy (from ~77% to ~90%) while simultaneously reducing racial bias.
The "Fairness-Accuracy" Paradox in Judicial ML
The primary criticism of tools like COMPAS is their tendency to "over-jail" certain populations. In technical terms, this is a disparate False Positive Rate: African Americans who would not have re-offended are incorrectly flagged as high-risk at a much higher rate than their Caucasian counterparts.
Previous works often struggled because they used limited "snapshots" of data (e.g., only looking at a specific release year). The authors hypothesized that the "signal" for future behavior isn't just in who a person is (demographics), but in the specific patterns of their past actions across many years.
Methodology: The Power of Rolling History
Instead of treating an offender's entire life as one data point, the researchers split the data into individual cycles.
1. Data Granularity
They expanded 38,624 records into 442,000 data points. Each point represents a moment in time, looking back at a "rolling window" of N previous arrests (10, 20, 40, 60, 80, 100 cycles).
2. Feature Engineering
They categorized history into 26 specific crime types. This allows the model to capture "Crime Specialization"—the tendency of individuals to persist in specific types of offenses (e.g., property vs. drug crimes).
Figure: The 4-step workflow from raw data split to FPR-based model selection.
3. Selection via FPR Parity
The team didn't just pick the most accurate model. They looked for the "Sweet Spot"—the model where the FPR Ratio (Disadvantaged Group FPR / Privileged Group FPR) approached 1.0. This ensures that the cost of being "wrongly accused" is distributed equally across races.
Deep Learning Results
The researchers utilized a 3-layer Artificial Neural Network (ANN) to map these complex histories to future outcomes.
Figure: The ANN architecture used to ingest rolling sums and personal details.
Key Findings:
- All Crimes: The 20-prior-arrest model was the winner. It achieved 89.9% accuracy and brought the FPR ratio down to 1.21 (down from 1.32 in the baseline).
- Sexual Crimes: The model achieved an impressive 1.07 FPR ratio, indicating near-perfect racial parity in prediction errors for this category.
SOTA Comparison
Compared to previous benchmarks (Ozkan, 2017) using the same dataset, this method improved sensitivity (True Positive Rate) while maintaining significantly lower error rates.
Figure: Accuracy and Bias metrics across different cycle lengths for Sexual Crimes.
Critical Insight & Perspective
The genius of this work lies in "Fairness through Unawareness + Rich Context." By removing the "Race" variable entirely but providing the model with a dense, 26-category history of behavior, the model learns to identify risk patterns rather than racial proxies.
Limitations: The authors acknowledge that the dataset (1994) is aged. Additionally, while the FPR ratio improved, it was not eliminated. The Caucasian group still has a lower FPR than the African American group in the "All Crimes" category, likely reflecting historical biases already "baked" into sentencing and conviction records.
Conclusion
This paper is a blueprint for "Responsible AI" in social sciences. It demonstrates that by increasing the temporal resolution of data—looking at the sequence of events rather than a static profile—we can build systems that are significantly more accurate and demonstrably fairer.