Abstract
Sophisticated malware frequently employs advanced evasion techniques to remain undetected by traditional security mechanisms. One of the most commonly used tactics is process injection, where malicious code is covertly inserted into the address space of legitimate processes. This allows the malware to operate under the guise of trusted applications, making detection significantly more challenging. In response to this issue, the present study introduces a novel detection methodology that functions entirely outside the virtual machine (out-of-VM). This technique leverages advanced memory introspection to identify and analyze different forms of process injection within virtualized environments. Notably, the approach is agentless, meaning it does not require any software to be installed within the guest VM, thereby eliminating the risk of the detection system itself being compromised or bypassed by the malware. Instead, it analyzes memory from the hypervisor level, providing a more secure and isolated vantage point. Experimental evaluations validate the effectiveness of the proposed method, demonstrating superior performance when compared to existing detection frameworks. Specifically, the method achieves higher detection accuracy, with more true positives and fewer false positives. It is capable of precisely identifying injected memory regions and detecting a broader spectrum of malware types, thereby outperforming current state-of-the-art solutions across all major evaluation metrics.
Keywords: Malware Detection, Memory Analysis, Process Injection, Security, Virtual Machine Introspection, Volatility, Windows.