TEA Framework: Bridging the "Label Gap" in Clinical Machine Learning

TL;DR

The Temporal Event Annotator (TEA) is an open-source mHealth framework that automates the creation of customized mobile apps for real-time clinical event logging. By using a schema-driven architecture, it allows researchers to define complex event ontologies—such as NICU interventions or patient stress indicators—and instantly generate a tablet-optimized UI for field researchers to record "gold-standard" labels for AI training.

The Motivation: From Paper Logs to Machine-Readable Gold Standards

In the race to build predictive models for healthcare, we have plenty of "input" data (heart rates, ECG, video) but a severe shortage of "output" labels (exactly when did the patient cough? when was the diaper changed?).

Traditional methods are fundamentally broken:

• Pen and Paper: Impossible to synchronize with millisecond-accurate sensor data.
• Fixed Apps: Require a developer to rewrite the code every time a doctor wants to track a new type of clinical event.
• Census Tools: General survey apps (like Survey123) aren't built for the "stop-start" nature of continuous temporal events.

Methodology: The Power of Schemas

The TEA framework's core innovation lies in its Ontology-Driven Architecture. Instead of hard-coding buttons, the app "builds itself" by reading two JSON-style schemas:

1. Terminology Schema: Defines the what. It structures the data into Sessions, Event Categories, and Events (both instantaneous and continuous).
2. UI Schema: Defines the how. It dictates whether a category appears as a grid or a list, which icons are used, and the order of buttons based on the frequency of clinical use.

Architecture and Implementation

The system follows a classic Client-Server-Mobile triad:

• TEA API: A NodeJS/Express service that handles authentication (HMAC) and data persistence in MongoDB.
• TEA Fabric (Mobile): An Android-based interface optimized for 10-inch tablets. Crucially, it supports offline usage, automatically syncing data once a hospital Wi-Fi connection is restored.
• TEA Central (Web): A dashboard for researchers to design their studies and export data in CSV/JSON for model training.

Real-World Validation: From NICU to VR

The authors proved the framework's versatility through three distinct clinical use cases:

• Case 1: NICU Monitoring: In a study at CHEO, researchers tracked 15 sessions of neonatal care. The app handled a massive ontology including 12 types of routine care, 10 physiological events, and even 9 specific types of pressure applied to a specialized mat.
• Case 2: VR Stress Estimation: In psychiatric therapy sessions, clinicians used the app to mark signs of "Sympathetic Activation" (SAANS) like skin flushing or rapid eye movement, which were then used to label motion capture and heart rate data.
• Case 3: Transport Vibration: In the chaotic environment of an emergency transport vehicle, the app replaced stopwatches and pencils, allowing researchers to mark "speed bumps" and "highway entry" events with a single tap.

Critical Insight: The "Sync" Problem

One of the most practical takeaways from the paper is the "Sync Event." If a medical sensor and a tablet are not on the same network clock, synchronization fails. The TEA framework suggests a physical "Sync" event—e.g., shaking an accelerometer while simultaneously pressing a "Sync" button in the app—to create a distinct "spike" in both data streams, allowing for perfect alignment in post-processing.

Conclusion and Future Work

The TEA framework moves mHealth annotation from a "software engineering task" to a "data design task." By allowing clinicians to update ontologies in real-time without coding, it significantly lowers the barrier to creating high-quality datasets for medical AI. Future iterations could benefit from Cross-platform support (iOS/Flutter) and perhaps voice-integrated annotation to allow clinicians to keep their hands free during critical care.

Key Values:

• Dynamic: No redeployment needed for new event types.
• Robust: Local-first storage ensures no data loss during hospital Wi-Fi dead zones.
• Scalable: Decouples the clinical definitions from the technical infrastructure.