BI-Driven Risk Analysis: Uncovering Hidden Patterns in Colombian Diabetes Care

Executive Summary

TL;DR: This research develops a Business Intelligence (BI) approach to centralize clinical data from Colombian health entities, applying data mining to discover risk factors for Type 2 Diabetes (DM2). By utilizing Association Rules, the authors identified unexpected links between DM2 and respiratory/gastric conditions, providing a roadmap for more effective preventative care.

Positioning: This work serves as an application-oriented bridge between data engineering (warehousing) and clinical decision support, moving beyond basic descriptive statistics to "intelligent" pattern discovery in a specific regional context.

Problem & Motivation: The Cost of Fragmented Data

In Colombia, chronic diseases like DM2 represent a massive financial burden, with treatment costs often doubling those of non-diabetic patients.

The authors identify a critical gap: while vast amounts of data are collected for administrative purposes (billing and insurance), they remain siloed and underutilized for clinical risk analysis. Traditional regression models used by health providers often fail to capture complex, multi-variable relationships (hidden patterns) that exist within large populations. The motivation was to turn "dead data" into a dynamic tool for early intervention.

Methodology: The BI Pipeline

The authors implemented a multi-stage workflow to transform raw clinical records into knowledge:

1. Centralized Clinical Data Warehouse

The architecture integrates:

• Demographics: Age, sex, and socio-economic variables.
• Clinical Diagnoses: Standardized via CIE10 (International Classification of Diseases).
• Procedures: Standardized via CUPS (Colombian classification).
• Eco-sociocultural Factors: Including smoking status, education, and arterial pressure.

2. Association Rule Mining

The study employed two primary algorithms:

• Apriori: Used for general demographic/behavioral patterns (Dataset A).
• FP-Growth: An optimized tree-based approach used for high-volume diagnosis and procedure data (Datasets B and C) to minimize computational overhead.

Figure 1: Distribution of eco-sociocultural variables across the study population.

Experiments & Results: Beyond Clinical Guidelines

The analysis was performed on a dataset of 21,936 DM2 patients. Setting a confidence threshold of 0.85, several key insights emerged:

• The Gender/Age Gap: Women between 52 and 65 years old showed an extremely high confidence (0.98) in attending follow-up exams (Glycosylated Hemoglobin) specifically in July, suggesting a seasonal behavior in healthcare utilization among this demographic.
• Unexpected Co-morbidities: The algorithms flagged a strong association between DM2 and Chronic Obstructive Pulmonary Disease (COPD) (Confidence: 0.92) and Gastritis. While DM2 clinical guidelines usually focus on cardiovascular and renal risks, these results suggest that respiratory health and adverse drug reactions (causing gastritis) are significant, overlooked factors in the Colombian population.
• Operational Validation: The system identified that while many patients receive standard care (creatinine, potassium tests), a high number of "Chest X-rays" are being ordered alongside diabetic controls, prompting a need for the insurer to audit procedural necessity.

Figure 2: Association rules showing high confidence between Gastritis, COPD, and DM2.

Critical Analysis & Conclusion

Takeaway

The study proves that BI isn't just for "business" (finance/marketing)—in healthcare, it is a clinical tool. By centralizing data and using association rules, providers can move from reactive treatment to proactive population management.

Limitations

• Data Sparsity: Many eco-sociocultural variables (like smoking history) had low population rates, limiting the depth of lifestyle-based rules.
• Directional Causality: Association rules indicate correlation ($X\to Y$), but they do not inherently prove that DM2 causes the associated conditions; clinical expert validation remains mandatory.

Future Outlook

The next step for this line of research is moving from descriptive association rules to predictive classification (e.g., Random Forests or Neural Networks) to identify which healthy patients are most likely to develop DM2 complications before they occur.