Predicting Disease via Inter-Disease Correlation: A Data Mining Approach to Hypertension

TL;DR

Most medical prediction models rely on blood work and lab results. This study shifts the paradigm by asking: Can we predict hypertension purely based on the presence of other diseases? By leveraging a massive dataset of 300,000+ records, under-sampling to fix class imbalance, and an ensemble of J48 decision trees, the authors achieved an accuracy of 83.5%, proving that the "disease landscape" of a patient is a potent predictor of their condition.

The Problem: Data Imbalance and Missing Indices

In clinical settings, "missing data" is a feature, not a bug. Pathological indices like cholesterol or blood sugar aren't always available in every historical record. Furthermore, most patients in a database are healthy regarding a specific condition, creating a class-imbalanced scenario where a model might achieve 99% accuracy simply by predicting "healthy" for everyone.

This paper tackles two specific hurdles:

1. Feature Dependency: Moving away from lab results to "Presence/Absence" Boolean features.
2. Skewed Distributions: Using under-sampling to create balanced training sets that force the model to learn the characteristics of the diseased minority.

Methodology: Ensembles and Rough Sets

The authors extracted records for nine major conditions, including Diabetes, Septicemia, and Pneumonia, using Hypertension as the target variable.

1. The Classifier Pipeline

The team used Weka to deploy:

• Naive Bayes (NB): Highlighting probabilistic relationships.
• J48 (C4.5 Decision Tree): Providing interpretable rules.
• J48 Ensemble: A majority-vote system comprising five different trees trained on varying under-sampled subsets.

2. Architecture and Logic

The core "Insight" was the application of Rough Set Theory via the ROSE2 tool. This was used for "second-order learning"—treating the binary outputs of the ensemble members as new features to identify "reducts" (the minimum subset of classifiers needed to maintain accuracy).

(Formulaic definitions of evaluation metrics used to validate the model's performance on imbalanced data)

Experiments & SOTA Results

The results confirm that ensemble methods significantly "catch" more positive cases (Sensitivity). While single models like NB and J48 achieved ~83% accuracy, the ensemble pushed metrics across the board.

Note: The J48 trees often skewed based on specific high-correlation diseases like Congestive Heart Failure or Diabetes.

Key Finding: Ensemble Reduction

By applying Rough Set tools, the authors found that they could reduce the ensemble from 5 classifiers down to 3 without a significant drop in performance. This is critical for deploying models in resource-constrained Healthcare Information Systems (HIS).

Critical Analysis & Future Outlook

Takeaway

The study demonstrates that clinical diagnosis doesn't always need "deep" lab metrics to be effective. The inter-connectivity of diseases (the "Comorbidity Network") provides enough signal for significant predictive power.

Limitations & Future Work

• The "Minor" Improvement: While the Sensitivity gains are impressive, the overall Accuracy improvement was incremental (0.46%). This suggests a potential saturation point for binary disease features.
• Next Steps: Future researchers should integrate pathological data with these inter-disease relationships to create a "hybrid" model that balances the breadth of Boolean features with the depth of lab results.

Conclusion

This work serves as a foundational bridge between traditional medical statistics and modern data mining. By successfully handling class imbalance and optimizing ensembles through Rough Set theory, it provides a roadmap for more efficient, automated diagnostic support systems in modern hospitals.