From Flow to Packet: A Unified Machine Learning Approach for Advanced Intrusion Detection
Didik Sudyana
Computer and Network Center , National Cheng Kung University , Tainan , 701 , Taiwan , ncku.edu.tw
Search for more papers by this authorFietyata Yudha
Department of Computer Science , National Yang Ming Chiao Tung University , Hsinchu , 300 , Taiwan , nctu.edu.tw
Search for more papers by this authorCorresponding Author
Ying-Dar Lin
Department of Computer Science , National Yang Ming Chiao Tung University , Hsinchu , 300 , Taiwan , nctu.edu.tw
Search for more papers by this authorChia-Hung Lai
Industrial Technology Research Institute , Hsinchu , 310 , Taiwan , itri.org.tw
Search for more papers by this authorPo-Ching Lin
Department of Computer Science and Information Engineering , National Chung Cheng University , Chiayi , 600 , Taiwan , ccu.edu.tw
Search for more papers by this authorRen-Hung Hwang
College of Artificial Intelligence , National Yang Ming Chiao Tung University , Tainan , 710 , Taiwan , nctu.edu.tw
Search for more papers by this authorDidik Sudyana
Computer and Network Center , National Cheng Kung University , Tainan , 701 , Taiwan , ncku.edu.tw
Search for more papers by this authorFietyata Yudha
Department of Computer Science , National Yang Ming Chiao Tung University , Hsinchu , 300 , Taiwan , nctu.edu.tw
Search for more papers by this authorCorresponding Author
Ying-Dar Lin
Department of Computer Science , National Yang Ming Chiao Tung University , Hsinchu , 300 , Taiwan , nctu.edu.tw
Search for more papers by this authorChia-Hung Lai
Industrial Technology Research Institute , Hsinchu , 310 , Taiwan , itri.org.tw
Search for more papers by this authorPo-Ching Lin
Department of Computer Science and Information Engineering , National Chung Cheng University , Chiayi , 600 , Taiwan , ccu.edu.tw
Search for more papers by this authorRen-Hung Hwang
College of Artificial Intelligence , National Yang Ming Chiao Tung University , Tainan , 710 , Taiwan , nctu.edu.tw
Search for more papers by this authorAbstract
In the era of advanced networking with 5G integration, the need for efficient and scalable intrusion detection systems has become critical to securing large-scale digital infrastructures. Traditional intrusion detection approaches either analyze individual packets yielding high computational costs or rely solely on flow-based data, which can miss important sequence-level information critical to identifying interservice communications and attack behaviors. To address this, we propose a unified machine learning approach that integrates flow-based and packet-based detection using convolutional neural networks (CNNs) for advanced intrusion detection. Our method prioritizes flow-based detection for short flows as the first defense layer and selectively invokes packet-based detection for longer flows or cases deemed uncertain. Uncertain predictions from the flow-based stage are identified using a confidence threshold and re-evaluated by the packet-based system. We validate our method using a systematically generated dataset from a microservices environment alongside benchmark datasets, including CIC-IDS-2017, CIC-IDS-2018, and CREMEv2. This hybrid detection strategy yields strong performance in both accuracy and efficiency. Specifically, our approach reduces the computational cost by up to 24 × (approximately 1.38 orders of magnitude) compared to relying solely on packet-based analysis. Additionally, the model demonstrates strong generalization with detection rates of 95% and 100% for flow- and packet-based detection, respectively, even against previously unseen attacks generated through behavioral variations and command-level perturbations.
Conflicts of Interest
The authors declare no conflicts of interest.
Open Research
Data Availability Statement
The data that support the findings of this study are available on request from the first author. The data are not publicly available due to privacy or ethical restrictions.
References
- 1 Agiwal M., Roy A., and Saxena N., Next Generation 5G Wireless Networks: A Comprehensive Survey, IEEE Communications Surveys & Tutorials. (2016) 18, no. 3, 1617–1655, https://doi.org/10.1109/COMST.2016.2532458, 2-s2.0-84984848251.
- 2
Rezaei Nasab A.,
Shahin M.,
Hoseyni Raviz S. A.,
Liang P.,
Mashmool A., and
Lenarduzzi V., An Empirical Study of Security Practices for Microservices Systems, Journal of Systems and Software. (2023) 198, https://doi.org/10.1016/j.jss.2022.111563.
10.1016/j.jss.2022.111563 Google Scholar
- 3 Liu J., Song X., Zhou Y. et al., Deep Anomaly Detection in Packet Payload, Neurocomputing. (2022) 485, 205–218, https://doi.org/10.1016/j.neucom.2021.01.146.
- 4
Marn G.,
Casas P., and
Capdehourat G., RawPower: Deep Learning Based Anomaly Detection from Raw Network Traffic Measurements, Proceedings of the ACM SIGCOMM 2018 Conference on Posters and Demos. SIGCOMM ’18, 2018, Budapest, Hungary, Association for Computing Machinery, 75–77, https://doi.org/10.1145/3234200.3234238, 2-s2.0-85056455475.
10.1145/3234200.3234238 Google Scholar
- 5 Wang W., Sheng Y., Wang J. et al., HAST-IDS: Learning Hierarchical Spatial-Temporal Features Using Deep Neural Networks to Improve Intrusion Detection, IEEE Access. (2018) 6, 1792–1806, https://doi.org/10.1109/ACCESS.2017.2780250, 2-s2.0-85038876745.
- 6
Lotfollahi M.,
Jafari Siavoshani M.,
Zade R. S. H. et al., Deep Packet: A Novel Approach for Encrypted Traffic Classification Using Deep Learning, Soft Computing. (2020) 24.4, 1999–2012, https://doi.org/10.1007/s00500-019-04030-2, 2-s2.0-85065760726.
10.1007/s00500-019-04030-2 Google Scholar
- 7 Miyamoto K., Iida M., Han C., Ban T., Takahashi T., and Takeuchi J., Consolidating Packet-Level Features for Effective Network Intrusion Detection: A Novel Session-Level Approach, IEEE Access. (2023) 11, 132792–132810, https://doi.org/10.1109/ACCESS.2023.3335600.
- 8 Lan J., Liu X., Li B., Sun J., Li B., and Zhao J., MEMBER: A Multi-Task Learning Model with Hybrid Deep Features for Network Intrusion Detection, Computers & Security. (2022) 123, https://doi.org/10.1016/j.cose.2022.102919.
- 9
Fox G. and
Boppana R. V., Detection of Malicious Network Flows With Low Preprocessing Overhead, Network. (2022) 2, no. 4, 628–642, https://doi.org/10.3390/network2040036.
10.3390/network2040036 Google Scholar
- 10 He K., Kim D. D., and Asghar M. R., Adversarial Machine Learning for Network Intrusion Detection Systems: A Comprehensive Survey, IEEE Communications Surveys & Tutorials. (2023) 25, no. 1, 538–566, https://doi.org/10.1109/COMST.2022.3233793.
- 11 Halbouni A., Gunawan T. S., Habaebi M. H., Halbouni M., Kartiwi M., and Ahmad R., Machine Learning and Deep Learning Approaches for CyberSecurity: A Review, IEEE Access. (2022) 10, 19572–19585, https://doi.org/10.1109/access.2022.3151248.
- 12
Zhou Y.,
Shi H.,
Zhao Y.,
Gao W., and
Zhang W., Encrypted Network Traffic Identification Based on 2D-CNN Model, 2021 22nd Asia-Pacific Network Operations and Management Symposium (APNOMS), 2021, 238–241, https://doi.org/10.23919/APNOMS52696.2021.9562636.
10.23919/APNOMS52696.2021.9562636 Google Scholar
- 13 Niu W., Zhuo Z., Zhang X., Du X., Yang G., and Guizani M., A Heuristic Statistical Testing Based Approach for Encrypted Network Traffic Identification, IEEE Transactions on Vehicular Technology. (2019) 68, no. 4, 3843–3853, https://doi.org/10.1109/TVT.2019.2894290, 2-s2.0-85064666463.
- 14 Bu Z., Zhou B., Cheng P., Zhang K., and Ling Z.-H., Encrypted Network Traffic Classification Using Deep and Parallel Network-In-Network Models, IEEE Access. (2020) 8, 132950–132959, https://doi.org/10.1109/ACCESS.2020.3010637.
- 15 Vinayakumar R., Soman K. P., Poornachandran P., and Akarsh S., Application of Deep Learning Architectures for Cyber Security, Advanced Sciences and Technologies for Security Applications, 2019.
- 16
Wang W.,
Zhu M.,
Wang J.,
Zeng X., and
Yang Z., End-to-End Encrypted Traffic Classification With One-Dimensional Convolution Neural Networks, 2017 IEEE International Conference on Intelligence and Security Informatics (ISI), 2017, 43–48, https://doi.org/10.1109/ISI.2017.8004872, 2-s2.0-85030236108.
10.1109/ISI.2017.8004872 Google Scholar
- 17
Okonkwo Z.,
Foo E.,
Li Q., and
Hou Z., A CNN Based Encrypted Network Traffic Classifier, ACSW’22, 2022, Association for Computing Machinery, Brisbane, Australia, 74–83, https://doi.org/10.1145/3511616.3513101.
10.1145/3511616.3513101 Google Scholar
- 18 Wang B., Su Y., Zhang M., and Nie J., A Deep Hierarchical Network for Packet-Level Malicious Traffic Detection, IEEE Access. (2020) 8, 201728–201740, https://doi.org/10.1109/ACCESS.2020.3035967.
- 19
Ullah I. and
Mahmoud Q. H., A Two-Level Hybrid Model for Anomalous Activity Detection in IoT Networks, 2019 16th IEEE Annual Consumer Communications Networking Conference (CCNC), 2019, 1–6, https://doi.org/10.1109/CCNC.2019.8651782, 2-s2.0-85063504474.
10.1109/CCNC.2019.8651782 Google Scholar
- 20
Bovenzi G.,
Aceto G.,
Ciuonzo D.,
Persico V., and
Pescape A., A Hierarchical Hybrid Intrusion Detection Approach in IoT Scenarios, GLOBECOM 2020-IEEE Global Communications Conference, 2020, 1–7, https://doi.org/10.1109/GLOBECOM42002.2020.9348167.
10.1109/GLOBECOM42002.2020.9348167 Google Scholar
- 21 Ashfaq Khan M., Rezaul Karim Md., and Kim Y., A Scalable and Hybrid Intrusion Detection System Based on the Convolutional-LSTM Network, Symmetry 11.4, 2019, https://doi.org/10.3390/sym11040583, 2-s2.0-85065494532.
- 22 Verkerken M., D’hooge L., Sudyana D. et al., A Novel Multi-Stage Approach for Hierarchical Intrusion Detection, IEEE Transactions on Network and Service Management. (2023) 20, no. 3, 3915–3929, https://doi.org/10.1109/TNSM.2023.3259474.
- 23
Rueda D. F.,
Caviedes J. C., and
Yesid Campo Muñoz W., A. Pasumpon Pandian, X. Fernando, and W. Haoxiang, A Hybrid Intrusion Detection Approach Based on Deep Learning Techniques, Computer Networks, Big Data and IoT, 2022, Springer Nature Singapore, Singapore, 863–878.
10.1007/978-981-19-0898-9_65 Google Scholar
- 24
Ammar A., Toward Efficient Intrusion Detection System Using Hybrid Deep Learning Approach, Symmetry. (2022) 14.9, https://doi.org/10.3390/sym14091916.
10.3390/sym14091916 Google Scholar
- 25 Bamber S. S., Katkuri A. V. R., Sharma S., and Angurala M., A Hybrid CNN-LSTM Approach for Intelligent Cyber Intrusion Detection System, Computers & Security. (2025) 148, https://doi.org/10.1016/j.cose.2024.104146.
- 26 Mahdi Z. S., Zaki R. M., and Alzubaidi L., Advanced Hybrid Techniques for Cyberattack Detection and Defense in IoT Networks, Security and Privacy, 2024, https://doi.org/10.1002/spy2.471.
- 27 Zulfiqar Z., Malik S. U. R., Moqurrab S. A., Zulfiqar Z., Yaseen U., and Srivastava G., DeepDetect: An Innovative Hybrid Deep Learning Framework for Anomaly Detection in IoT Networks, Journal of Computational Science. (2024) 83, https://doi.org/10.1016/j.jocs.2024.102426.
- 28
Koca M. and
Avci I., A Novel Hybrid Model Detection of Security Vulnerabilities in Industrial Control Systems and IoT Using GCN+LSTM, IEEE Access. (2024) 12, 143343–143351, https://doi.org/10.1109/ACCESS.2024.3466391.
10.1109/ACCESS.2024.3466391 Google Scholar
- 29 Alsaffar A. M., Nouri-Baygi M., and Zolbanin H. M., Shielding Networks: Enhancing Intrusion Detection with Hybrid Feature Selection and Stack Ensemble Learning, Journal of Big Data. (2024) 11, no. 1, https://doi.org/10.1186/s40537-024-00994-7.
- 30 Cai S., Han D., Yin X., Li D., and Chang C.-C., A Hybrid Parallel Deep Learning Model for Efficient Intrusion Detection Based on Metric Learning, Connection Science. (2022) 34, no. 1, 551–577, https://doi.org/10.1080/09540091.2021.2024509.
- 31 Garg S., Kaur K., Kumar N., Kaddoum G., Zomaya A. Y., and Ranjan R., A Hybrid Deep Learning-Based Model for Anomaly Detection in Cloud Datacenter Networks, IEEE Transactions on Network and Service Management. (2019) 16, no. 3, 924–935, https://doi.org/10.1109/TNSM.2019.2927886, 2-s2.0-85069501526.
- 32 GoogleCloudPlatform, Google Microservices Demo, https://github.com/GoogleCloudPlatform/microservices-demo.
- 33
Gan Y.,
Zhang Y.,
Cheng D. et al., An Open-Source Benchmark Suite for Microservices and Their Hardware-Software Implications for Cloud Amp; Edge Systems, ASPLOS’19, 2019, Association for Computing Machinery, Providence, 3–18, https://doi.org/10.1145/3297858.3304013, 2-s2.0-85064688619.
10.1145/3297858.3304013 Google Scholar
- 34
Jain T. and
Jain N., Framework for Web Application Vulnerability Discovery and Mitigation by Customizing Rules through ModSe-Curity, 2019 6th International Conference on Signal Processing and Integrated Networks (SPIN), 2019, 643–648, https://doi.org/10.1109/SPIN.2019.8711673, 2-s2.0-85066883438.
10.1109/SPIN.2019.8711673 Google Scholar
- 35 Antirez, Antirez/hping: Hping Network Tool, https://github.com/antirez/hping.
- 36 Yaltirakli G., Slowloris, github.com, 2015, https://github.com/gkbrk/slowloris.
- 37 Sharafaldin I., Lashkari A. H., and Ghorbani A. A., Toward Generating a New Intrusion Detection Dataset and Intrusion Traffic Characterization, International Conference on Information Systems Security and Privacy, 2018.
- 38 Yudha F., Lin Y.-D., Lai Y.-C., Sudyana D., and Hwang R.-H., Reproducing ATTCK Techniques and Lifecycles to Train Machine Learning Classifier, IEEE Network, 2025, https://doi.org/10.1109/MNET.2025.3551333.
- 39 Aouini Z. and Pekar A., NFStream: A Flexible Network Data Analysis Framework, Computer Networks. (2022) 204, https://doi.org/10.1016/j.comnet.2021.108719.
- 40 Hwang R.-H., Peng M.-C., Huang C.-W., Lin Po-C., and Nguyen V.-L., An Unsupervised Deep Learning Model for Early Network Traffic Anomaly Detection, IEEE Access. (2020) 8, 30387–30399, https://doi.org/10.1109/ACCESS.2020.2973023.
- 41
Mohammadpour L.,
Chaw Ling T.,
Sun Liew C., and
Aryanfar A., A Survey of CNN-Based Network Intrusion Detection, Applied Sciences, 2022, https://doi.org/10.3390/app12168162.
10.3390/app12168162 Google Scholar
- 42
Akiba T.,
Sano S.,
Yanase T.,
Ohta T., and
Koyama M., Optuna: A Next-Generation Hyperparameter Optimiza-tion Framework, Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. KDD’19, 2019, Anchorage, AK, USA, Association for Computing Machinery, 2623–2631, https://doi.org/10.1145/3292500.3330701, 2-s2.0-85071168537.
10.1145/3292500.3330701 Google Scholar