Preempting the Invisible: Early In-Memory Malware Detection via Run-Time Semantics

TL;DR

Researchers from Ben-Gurion University have developed a novel end-to-end system that detects in-memory attacks before they can execute harmful payloads. By treating runtime environmental features—like stack snapshots and register states—as "language," they achieved a staggering 99.98% accuracy with an ultra-low false positive rate, making it viable for high-stakes production environments.

Problem & Motivation: The Ghost in the Machine

The security industry is currently facing a "fileless" crisis. Modern malware often resides entirely in a process's volatile memory, bypassing signature-based scanners that look for malicious files on the disk.

While sophisticated EDR (Endpoint Detection and Response) tools exist, they face a "trilemma":

1. Performance: Deep memory scanning is CPU-intensive.
2. Accuracy: Heuristic-based rules often flag legitimate administrative tools (False Positives).
3. Timeliness: Many tools detect malware after it has started encrypted files or exfiltrated data.

The authors' insight was to move away from looking for signatures and instead look at the environment. If a process's runtime environment (its stack, its loaded modules, its registers) looks "wrong" semantically, it’s a precursor to an attack.

Methodology: Turning System Logs into Semantic Vectors

The core innovation lies in the transformation of raw system data into a format machine learning can understand.

1. Feature Selection

The authors extracted 40 specific features, categorized into:

• Stack Snapshots & Traces: What is the calling convention?
• Register Data: What opcodes are currently in memory?
• Loaded Modules: Are there suspicious DLLs or unusual memory mappings?

2. The NLP Pipeline (Word2Vec)

Instead of manually engineering rules for what a "bad" register value looks like, the authors used Word2Vec. By treating each unique feature value as a "word," the model learns the context of these values. For instance, a specific memory offset used by a Trojan will appear in a similar "semantic context" as other known malicious offsets.

3. Architecture

The system uses a split architecture to ensure the endpoint stays fast:

• The Agent: A lightweight service on the user's machine that collects logs.
• The Detector: A cloud-based LightGBM model that processes the vector and returns a maliciousness score.

Figure 1: The High-Level Flow of the Log Extraction and Detection Process.

Experiments & Results: SOTA Performance

The model was trained and tested on a massive dataset of over 2.4 million logs provided by Morphisec Ltd, including 20 different malware categories.

Key Metrics

The results (using a classification threshold of 0.75) highlight the system's robustness:

• AUC: 0.9977
• Recall (TPR): 99.99% (Almost every attack was caught)
• FPR: 0.04% (Only 4 false alarms per 10,000 benign events)

Table 1: Performance metrics showcasing the high precision and recall of the proposed model.

The confusion matrix reveals that out of 301,213 actual malicious samples, only 11 were missed (False Negatives). This level of sensitivity is critical for preventing ransomware.

Critical Analysis & Conclusion

Takeaway

The paper proves that Run-Time Environmental Features are a goldmine for early detection. By shifting the focus from "what the file is" (Static Analysis) to "what the environment looks like during execution" (Contextual Analysis), the authors have found a way to catch fileless malware that typically evades EDRs.

Limitations & Future Work

While the results are impressive, the study identifies several real-world hurdles:

• Network Dependency: The cloud-based detection requires an internet connection. Future iterations aim for "offline" detection.
• Retraining Overhead: As malware evolves, the model needs retraining, which can be resource-heavy.
• Early Warning Interval: The authors plan to further quantify exactly how many seconds or minutes of warning the system provides before the "first harmful action" occurs.

In conclusion, this end-to-end ML solution offers a promising path toward invisible, highly accurate, and non-intrusive security that can finally keep pace with the evolution of in-memory threats.