Detecting Fileless Malware through Memory Forensics with Recurrent Neural Networks

Fileless malware is an increasingly stealthy cybersecurity threat that executes entirely in volatile memory, leveraging legitimate system utilities to evade traditional signature-based and static analysis defenses. Conventional machine learning approaches typically rely on static features or aggregated behavioral indicators, which fail to capture the temporal and execution order dependencies inherent in fileless attacks. This paper presents a detection framework integrating memory forensics with a recurrent neural network (RNN)–based temporal modeling. Sequences of forensic and behavioral features extracted from volatile memory capture transient process activity, memory-resident module behavior, and short-lived network interactions characteristic of fileless attacks. A comparative evaluation of multiple RNN architectures shows that a bidirectional LSTM achieves the best performance, reaching an accuracy of 0.93 with a recall of 1.00. Feature analysis identifies process behavior, memory-resident modules, and ephemeral network connections as strong indicators of fileless activity. These results demonstrate that behavioral memory features derived from volatile memory provide a more reliable basis for detecting evasive in-memory threats than static file-based approaches.

Detecting Fileless Malware through Memory Forensics with Recurrent Neural Networks Presentation