Performance Evaluation of Boosting Algorithms on Intrusion Detection System Based on Real-Time CICIoT2023 Dataset

Muhammad Hafiz Amrullah; Favian Dewanta; Muhamad Erza Aminanto

doi:10.31436/iiumej.v27i3.4534

Authors

Muhammad Hafiz Amrullah Telkom University
Favian Dewanta Telkom University https://orcid.org/0000-0001-8427-8648
Muhamad Erza Aminanto Monash University, Indonesia https://orcid.org/0000-0001-5614-2276

DOI:

https://doi.org/10.31436/iiumej.v27i3.4534

Keywords:

Internet of Things (IoT), Machine Learning, Intrusion Detection System

Abstract

Currently, the utilization of machine learning methods for identifying intrusions in Internet of Things (IoT) networks demonstrates intriguing prospects. Choosing the appropriate machine learning technique for intrusion detection poses a significant challenge. Choosing the wrong algorithm may reduce threat-detection accuracy, increase the likelihood of network contamination, and compromise network security. This research conducts a comprehensive assessment of various boosting algorithms for detecting intrusions in IoT, using the latest CICIoT2023 dataset. The approach includes evaluating the effectiveness of these algorithms in both binary and multi-class classification scenarios. The evaluation includes a thorough analysis of classification performance metrics, including accuracy, precision, recall, and F1 score. It also assesses computational efficiency, particularly the duration of the training and testing procedures. The results show that the boosting algorithms have potential on the dataset used. Extreme Gradient Boosting yielded the highest accuracy for multi-class classification, whereas Categorical Boosting was the most effective for binary classification. Additionally, Categorical Boosting demonstrated superior test-time performance for multi-class classification, whereas Gradient Boosting excelled in binary classification.

ABSTRAK: Pada masa ini, penggunaan kaedah pembelajaran mesin bagi mengenal pasti pencerobohan rangkaian Internet Benda (IoT) menunjukkan potensi menarik. Memilih teknik pembelajaran mesin yang sesuai bagi mengesan pencerobohan menimbulkan cabaran ketara. Memilih algoritma yang salah boleh mengakibatkan ketepatan berkurangan dalam mengesan ancaman, kemungkinan pencemaran rangkaian yang lebih tinggi, dan keselamatan rangkaian terjejas. Kajian ini menjalankan penilaian komprehensif terhadap pelbagai algoritma penggalak yang digunakan dalam mengesan pencerobohan dalam IoT, menggunakan set data CICIoT2023 terkini. Pendekatan ini merangkumi penilaian keberkesanan algoritma ini dalam senario pengelasan binari dan berbilang kelas. Penilaian ini merangkumi analisis menyeluruh metrik prestasi pengelasan, termasuk ketepatan, penarikan balik, dan skor F1. Selain itu, ia melibatkan penilaian kecekapan pengiraan, terutama pada tempoh prosedur latihan dan pengujian. Keputusan menunjukkan bahawa algoritma penggalak mempunyai potensi pada set data yang digunakan. Pendekatan Galakan Kecerunan Ekstrem menghasilkan tahap ketepatan tertinggi bagi pengelasan berbilang kelas, manakala Galakan Kategori ditunjukkan sebagai yang paling berkesan bagi pengelasan binari. Di samping itu, algoritma Galakan Kategori menunjukkan prestasi masa ujian yang unggul bagi pengelasan berbilang kelas, manakala algoritma Galakan Kecerunan adalah terbaik pada pengelasan binari.

Downloads

Download data is not yet available.

Metrics

Metrics Loading ...

References

J. C. Cano, V. Berrios, B. Garcia and C. K. Toh, ”Evolution of IoT: An Industry Perspective,” in IEEE Internet of Things Magazine, vol. 1, no. 2, pp. 12-17, December 2018, doi: 10.1109/IOTM.2019.1900002.

A. Imteaj, U. Thakker, S. Wang, J. Li and M. H. Amini, ”A Survey on Federated Learning for Resource-Constrained IoT Devices,” in IEEE Internet of Things Journal, vol. 9, no. 1, pp. 1-24, 1 Jan.1, 2022, doi: 10.1109/JIOT.2021.3095077.

R. Ande, B. Adebisi, M. Hammoudeh, and J. Saleem, ”Internet of Things: Evolution and technologies from a security perspective”, Sustainable Cities and Society, vol. 54, p. 101728, 2020. doi: 10.1016/j.scs.2019.101728.

M. Almiani, A. AbuGhazleh, A. Al-Rahayfeh, S. Atiewi, and A. Razaque, ”Deep recurrent neural network for IoT intrusion detection system”, Simulation Modelling Practice and Theory, vol. 101, p. 102031, 2020. doi: 10.1016/j.simpat.2019.102031.

K. Ponnusamy and N. Rajagopalan, ”Internet of Things: A Survey on IoT Protocol Standards”, in Progress in Advanced Computing and Intelligent Engineering, 2018, pp. 651–663. doi: 10.1007/978-981-10-6875-1 64

S. Sathyadevan, K. Achuthan, R. Doss and L. Pan, ”Protean Authentication Scheme – A Time-Bound Dynamic KeyGen Authentication Technique for IoT Edge Nodes in Outdoor Deployments,” in IEEE Access, vol. 7, pp. 92419-92435, 2019, doi: 10.1109/ACCESS.2019.2927818.

W. E. Mohammed Aziz Al Kabir and M. S. Sharif, ”Securing IoT Devices Against Emerging Security Threats: Challenges and Mitigation Techniques”, Journal of Cyber Security Technology, vol. 7, no. 4, pp. 199–223, 2023. doi: 10.1080/23742917.2023.2228053.

P. Mishra, V. Varadharajan, U. Tupakula and E. S. Pilli, ”A Detailed Investigation and Analysis of Using Machine Learning Techniques for Intrusion Detection,” in IEEE Communications Surveys & Tutorials, vol. 21, no. 1, pp. 686-728, Firstquarter 2019, doi:10.1109/COMST.2018.2847722.

H. Binder, O. Gefeller, M. Schmid, and A. Mayr, ”The evolution of boosting algorithms”, Methods of Information in Medicine, vol. 53, no. 06, pp. 419–427, 2014. doi: 10.3414/ME13-01-0122.

J. L. Leevy, J. Hancock, R. Zuech and T. M. Khoshgoftaar, ”Detecting Cybersecurity Attacks Using Different Network Features with LightGBM and XGBoost Learners,” 2020 IEEE Second International Conference on Cognitive Machine Intelligence (CogMI), Atlanta, GA, USA, 2020, pp. 190-197, doi: 10.1109/CogMI50398.2020.00032.

E. Ashraf, N. F. F. Areed, H. Salem, E. H. Abdelhay, and A. Farouk, ”FIDChain: Federated Intrusion Detection System for Blockchain-Enabled IoT Healthcare Applications”, Healthcare, vol. 10, no. 6, 2022. doi: 10.3390/healthcare10061110.

Garg, S., Kumar, V., Payyavula, S.R.,”Identification of internet of things (IoT) attacks using gradient boosting: a cross dataset approach,” Telematique 21(1), 6982–7012, 2022.

M. M. Khan and M. Alkhathami, ”Anomaly detection in IoT-based healthcare: machine learning for enhanced security”, Scientific Reports, vol. 14, no. 1, p. 5872, Mar. 2024. doi: 10.1038/s41598-024-56126-x.

E. C. P. Neto, S. Dadkhah, R. Ferreira, A. Zohourian, R. Lu, and A. A. Ghorbani, ”CICIoT2023: A Real-Time Dataset and Benchmark for Large-Scale Attacks in IoT Environment”, Sensors, vol. 23, no. 13, 2023. doi: 10.3390/s23135941.

M. M. Ahsan, M. A. P. Mahmud, P. K. Saha, K. D. Gupta, and Z. Siddique, ”Effect of Data Scaling Methods on Machine Learning Algorithms and Model Performance”, Technologies, vol. 9, no. 3, 2021. doi: 10.3390/technologies9030052.

M. Bach, A. Werner, and M. Palt, ”The Proposal of Undersampling Method for Learning from Imbalanced Datasets”, Procedia Computer Science, vol. 159, pp. 125–134, 2019. doi: 10.1016/j.procs.2019.09.167.

A. R. Hakim, K. Ramli, M. Salman, and E. R. Agustina, “Improving Model Performance for Predicting Exfiltration Attacks Through Resampling Strategies”, IIUMEJ, vol. 26, no. 1, pp. 420–436, Jan. 2025.

M. Saied, S. Guirguis, and M. Madbouly, ”A Comparative Study of Using Boosting-Based Machine Learning Algorithms for IoT Network Intrusion Detection”, International Journal of Computational Intelligence Systems, vol. 16, no. 1, p. 177, Nov. 2023. doi:10.1007/s44196-023-00355-x.

H. Zhang, B. Zhang, L. Huang, Z. Zhang, and H. Huang, ”An Efficient Two-Stage Network Intrusion Detection System in the Internet of Things”, Information, vol. 14, no. 2, 2023. doi: 10.3390/info14020077.