Memento: Transforming Wearables into Emotional Biographers via EEG

Memento: An Emotion Driven Lifelogging System with Wearables

2017-07-01
Shiqi Jiang, Pengfei Zhou, Zhenjiang Li, Mo Li
Summary
Problem
Method
Results
Takeaways
Abstract

Memento is an emotion-driven lifelogging system that integrates EEG sensors with smart glasses to automatically capture memorial moments. By leveraging Brain-Computer Interface (BCI) techniques, it triggers video recording based on the user's emotional arousal and achieves SOTA-level emotion tagging on wearable devices.

    ## TL;DR
    Memento is a pioneering lifelogging system that uses EEG (brainwave) sensors embedded in smart glasses to "sense" when you are having a memorable emotional experience. Instead of recording 24/7 or waiting for you to press a button, it automatically triggers a video log when it detects a significant change in your emotional state, effectively digitizing your life based on how you *feel* rather than just what you *see*.

    ## Background: The Limits of Passive Sensing
    Lifelogging has long been caught between two extremes:
    1. **Manual Logging**: High cognitive load; you often forget to record the most spontaneous moments.
    2. **Continuous Passive Logging**: Massive data redundancy and rapid battery drain.

    The authors of Memento realized that **memorable moments usually coincide with emotional changes**. By tapping into the user's internal mental state using Electroencephalography (EEG), they created a "proactive" system that logs only what matters.

    ## The Technical Challenge: Brainwaves in the Wild
    Measuring EEG on a laboratory couch is easy; measuring it while a user is walking down a street wearing smart glasses is a nightmare. The authors identify two primary "noise" sources:
    *   **Physiological Noise**: Eye blinks and jaw clenching (EMG) overlap with brain signals.
    *   **Motion Artifacts**: As the user moves, the EEG electrodes drift on the scalp, causing signal spikes.

    ### Strategy 1: Robust Preprocessing
    To combat this, Memento uses a specialized pipeline:
    *   **Band-pass Filtering**: Focusing on the 7-31 Hz range (Alpha and Beta bands) which are most indicative of mental states.
    *   **Kernel-based Correlation**: A time-domain approach to identify and "subtract" the specific electrical signature of eye blinks.
    *   **Sensor Fusion Quality Scoring**: It uses the built-in Accelerometer (IMU) to detect vigorous movement. If the signal is too noisy due to movement, the system intelligently drops the segment to save energy.

    ![System Architecture](https://cdn.atominnolab.com/wisdoc/images/20260613-21d0a83d-d478-4487-943b-e0ff98211d07/page_002_block_000.png)

    ## Methodology: Two-Phase Recognition
    The most brilliant aspect of Memento is its **Edge-Cloud Split**. Emotion recognition algorithms (like Support Vector Machines or Deep Neural Networks) are too heavy for a Google Glass battery.

    1.  **Phase 1 (On-Glass)**: The device calculates **Katz’s Fractal Dimension (FD)**. This is a lightweight mathematical measure of the "complexity" of the brainwave. A sudden spike in the FD trend signals an "Arousal" event, which triggers the camera.
    2.  **Phase 2 (Cloud)**: The raw features are uploaded to a private cloud. Here, a full classification is performed to tag the video with specific emotions (e.g., "Happy," "Surprised," "Sad").

    ![Arousal Trend Trigger](https://cdn.atominnolab.com/wisdoc/images/20260613-21d0a83d-d478-4487-943b-e0ff98211d07/page_004_block_002.png)

    ## Performance and Insights
    The system was validated using the **DEAP dataset** (a standard benchmark for emotion analysis). 

    *   **Accuracy**: The average RMS error was approximately 2.76. While not perfect, it is highly effective at catching the *direction* of emotional change.
    *   **User Satisfaction**: In a 6-hour "wild" study, 83% of the moments users later identified as "memorable" were successfully captured by Memento's automatic trigger.
    *   **Energy Efficiency**: The sensing and processing overhead on Google Glass added less than 200mW, proving that "always-on" mental state monitoring is feasible on current-gen hardware.

    ![User Expectation Comparison](https://cdn.atominnolab.com/wisdoc/images/20260613-21d0a83d-d478-4487-943b-e0ff98211d07/page_007_block_004.png)

    ## Critical Analysis & Conclusion
    Memento successfully shifts the paradigm of lifelogging from "Environment-Aware" to "Self-Aware." 

    **Limitations**:
    *   **Hardware Integration**: Currently uses a separate Muse headband and Google Glass; true commercial success requires integrated electrodes within the glass frames.
    *   **Social Privacy**: Automatic recording still faces social hurdles, though the "emotional trigger" makes it more purposeful than "always-on" cameras.

    **Future Outlook**:
    As wearable SoCs (System-on-Chips) integrate more **Low Power Units (LPUs)**, we can expect Memento's logic to move entirely on-device, potentially enabling real-time emotional feedback for mental health or performance coaching.

Find Similar Papers

Try Our Examples

  • Search for recent papers that utilize EEG or other physiological sensors to trigger proactive actions in Augmented Reality (AR) or smart glass environments.
  • What is the origin of the Arousal-Valence bipolar model in emotion recognition, and how have subsequent deep learning models improved upon the Fractal Dimension (FD) features used in this work?
  • Explore research that applies multi-sensor fusion (IMU + EEG) to solve the problem of motion artifacts in wearable Brain-Computer Interfaces during outdoor activities.
Contents
Memento: Transforming Wearables into Emotional Biographers via EEG
1. TL;DR
2. Background: The Limits of Passive Sensing
3. The Technical Challenge: Brainwaves in the Wild
3.1. Strategy 1: Robust Preprocessing
4. Methodology: Two-Phase Recognition
5. Performance and Insights
6. Critical Analysis & Conclusion