Investigating the Efficacy of Deep Learning Techniques in Detecting Advanced Persistent Threats (APTs)

Abstract:

Advanced Persistent Threats (APTs) pose significant challenges to cybersecurity due to their stealthy, multi-stage nature. Traditional detection methods struggle to identify such complex behaviors. This study investigates the effectiveness of deep learning (DL) techniques in detecting APTs. Two research questions guide the study: (i) How effective is a custom Transformer-based model compared to existing DL models in detecting APTs? (ii) To what extent have DL models contributed to APT detection between 2020 and 2025? A two-fold methodology was adopted: a meta-analysis of 30 peer-reviewed studies and an empirical evaluation of a Transformer model trained on the CIC-IDS2018 dataset. The model achieved 99.71% accuracy with macro-averaged F1-score, precision, and recall values above 0.70, demonstrating strong overall performance but revealing challenges in classifying rare attack types. The meta-analysis further identified key research gaps, including limited use of multi-class classification, underutilization of Transformers, and a need for real-world datasets. Findings show that Transformer-based architectures are not only viable but good at modeling complex feature dependencies and detecting sophisticated APT behaviours. This research highlights both the strengths and limitations of DL for cybersecurity and suggests future directions for improving robustness and scalability in real-world deployment.

• Periodical: Engineering Headway (Volume 37)
• Pages: 163-178
• DOI: https://doi.org/10.4028/p-mgOS5t
• Online since: March 2026
• Authors: Christopher Ubaka Ebelogu*, Israel Musa, Emmanuel Abbah, Donatus Onyedikachi Njoku, Ohunene Rahama Obadaki
• Keywords: Advanced Persistent Threats, Cybersecurity, Deep Learning, Multi-Class Classification, Transformer Model

References

[1] Imber, D. (2025, April 1). The latest cybercrime statistics (updated April 2025). AAG IT Services. https://aag-it.com/the-latest-cyber-crime-statistics/

[2] Sekti, B. A., Laksono, M. N. G., Anwar, N., Laksono, M. I. A., Tiaraputri, Z. A., Iswahyudi, R. T., & Yuhefizar, Y. (2025). Introduction to cyber defense in space missions. In Advanced cyber defense for space missions and operations: Concepts and applications (p.1–28). IGI Global Scientific Publishing.

DOI: 10.4018/979-8-3693-7939-4.ch001

[3] Ahmed, M., & Gaber, M. (2024). An investigation on cyber espionage ecosystem. Journal of Cyber Security Technology, 1–25.

DOI: 10.1080/23742917.2024.2399389

[4] Mutalib, N. H. A., Sabri, A. Q. M., Wahab, A. W. A., Abdullah, E. R. M. F., & AlDahoul, N. (2024). Explainable deep learning approach for advanced persistent threats (APTs) detection in cybersecurity: A review. Artificial Intelligence Review, 57(11), 297.

DOI: 10.1007/s10462-024-10890-4

[5] Hartono, B., Silalahi, F. D., & Muthohir, M. (2024). Transformers in cybersecurity: Advancing threat detection and response through machine learning architectures. Journal of Technology Informatics and Engineering, 3(3), 382–396.

DOI: 10.51903/jtie.v3i3.211

[6] Yu, K., et al. (2021). Securing critical infrastructures: Deep-learning-based threat detection in IIoT. IEEE Communications Magazine, 59(10), 76–82

DOI: 10.1109/MCOM.101.2001126

[7] Hassannataj, J., Joloudari, M. Haderbadi, A. Mashmool, M. Ghasemigol, S. S. Band and A. Mosavi, "Early Detection of the Advanced Persistent Threat Attack Using Performance Analysis of Deep Learning," in IEEE Access, vol. 8, pp.186125-186137, 2020.

DOI: 10.1109/ACCESS.2020.3029202

[8] Bakhiet, A. M., & Aly, S. A. (2024). Hybridizing base-line 2D-CNN model with cat swarm optimization for enhanced advanced persistent threat detection. In 2024 International Telecommunications Conference (ITC-Egypt) (p.596–601). IEEE

DOI: 10.1109/ITC-Egypt61547.2024.10620569

[9] Niu, W., Zhou, J., Zhao, Y., Zhang, X., Peng, Y., & Huang, C. (2022). Uncovering APT malware traffic using deep learning combined with time sequence and association analysis. Computers & Security, 120, 102809

DOI: 10.1016/j.cose.2022.102809

[10] Böge, E., Ertan, M. B., Alptekin, H., & Çetin, O. (2024). Unveiling cyber threat actors: A hybrid deep learning approach for behavior-based attribution. Digital Threats: Research and Practice

DOI: 10.1145/3676284

[11] Mazumder, S., Neogy, S., Sur, T., et al. (2025). A comparative assessment of deep learning for adaptable DDoS threat detection in cloud computing systems. SN Computer Science, 6, 80

DOI: 10.1007/s42979-024-03643-1

[12] Do Xuan, C., & Dao, M. H. (2021). A novel approach for APT attack detection based on a combined deep learning model. Neural Computing and Applications, 33(20), 13251–13264.

DOI: 10.1007/s00521-021-05952-5

[13] AL-Aamri, A. S., Abdulghafor, R., Turaev, S., Al-Shaikhli, I., Zeki, A., & Talib, S. (2023). Machine learning for APT detection. Sustainability, 15(18), 13820.

DOI: 10.3390/su151813820

[14] Amaru, Y., Wudali, P., Elovici, Y., & Shabtai, A. (2024). RAPID: Robust APT detection and investigation using context-aware deep learning. arXiv preprint. https://arxiv.org/abs/2406.05362

DOI: 10.1016/j.comnet.2025.111744

[15] Jia, Z., Xiong, Y., Nan, Y., Zhang, Y., Zhao, J., & Wen, M. (2024). MAGIC: Detecting advanced persistent threats via masked graph representation learning. In 33rd USENIX Security Symposium (USENIX Security 24) (p.5197–5214).

[16] Mamun, M., & Shi, K. (2021, October). DeepTaskAPT: Insider APT detection using task-tree-based deep learning. In 2021, IEEE 20th International Conference on Trust, Security, and Privacy in Computing and Communications (TrustCom) (p.693–700). IEEE.

DOI: 10.1109/trustcom53373.2021.00102

[17] Joloudari, J. H., Haderbadi, M., Mashmool, A., Ghasemigol, M., Band, S. S., & Mosavi, A. (2020). Early detection of the advanced persistent threat attack using performance analysis of deep learning. IEEE Access, 8, 186125–186137

DOI: 10.1109/ACCESS.2020.3029202

[18] Chen, J., Lan, X., Zhang, Q., Ma, W., Fang, W., & He, J. (2025). Defending against APT attacks in cloud computing environments using grouped multi-agent deep reinforcement learning. IEEE Internet of Things Journal

DOI: 10.1109/JIOT.2025.3542119

[19] Mei, Y., Han, W., Li, S., Lin, K., Tian, Z., & Li, S. (2024). A novel network forensic framework for advanced persistent threat attack attribution through deep learning. IEEE Transactions on Intelligent Transportation Systems, 25(9), 12131–12140

DOI: 10.1109/TITS.2024.3360260

[20] Cai, X., Zhang, H., Ahmed, C. M., & Koide, H. (2025). Detecting advanced persistent threat exfiltration with ensemble deep learning tree models and novel detection metrics. IEEE Access, 13, 81803–81822

DOI: 10.1109/ACCESS.2025.3567772

[21] Yu, S. (2022). Fast detection of advanced persistent threats for smart grids: A deep reinforcement learning approach. In ICC 2022 - IEEE International Conference on Communications (p.2676–2681). IEEE

DOI: 10.1109/ICC45855.2022.9838858

[22] Saheed, K., & Henna, S. (2023). Deep reinforcement learning for advanced persistent threat detection in wireless networks. In 2023, the 31st Irish Conference on Artificial Intelligence and Cognitive Science (AICS) (p.1–6). IEEE

DOI: 10.1109/AICS60730.2023.10470498

[23] Do Xuan, C., & Huong, D. T. (2022). A new approach for APT malware detection based on deep graph network for endpoint systems. Applied Intelligence, 52(12), 14005–14024.

DOI: 10.1007/s10489-021-03138-z

[24] Do Xuan, C., & Duong, D. (2022). Optimization of APT attack detection based on a model combining ATTENTION and deep learning. Journal of Intelligent & Fuzzy Systems, 42(4), 4135–4151.

DOI: 10.3233/jifs-212570

[25] Do Xuan, C., Dao, M. H., & Nguyen, H. D. (2020). APT attack detection based on flow network analysis techniques using deep learning. Journal of Intelligent & Fuzzy Systems, 39(3), 4785–4801.

DOI: 10.3233/jifs-200694

[26] Tang, B., Yang, J., Li, X., Cao, Y., & Wang, J. (2023). APT detector: Detect and identify APT malware based on deep learning framework. In Proceedings of the 2023 9th International Conference on Computing and Artificial Intelligence

DOI: 10.1145/3594315.3594374

[27] Mansour Bahar, A. A., Ferrahi, K. S., Messai, M.-L., Seba, H., & Amrouche, K. (2024). FedHE-graph: Federated learning with hybrid encryption on graph neural networks for advanced persistent threat detection. In Proceedings of the 19th International Conference on Availability, Reliability and Security (p.1–10)

DOI: 10.1145/3664476.3670466

[28] Alrehaili, M., Alshamrani, A., & Eshmawi, A. (2021). A hybrid deep learning approach for advanced persistent threat attack detection. In The 5th International Conference on Future Networks & Distributed Systems

DOI: 10.1145/3508072.3508085