Abstract N-BaIoT is one of the most widely used datasets for detecting botnet attacks on Internet of Things (IoT) devices, and published classifiers routinely report near-perfect accuracy on it. This paper shows that ten of its 115 features, the mean and variance of the host-to-host jitter statistic (HH_jit) at five time windows, frequently hold Unix timestamps instead of traffic statistics. When a channel has seen a single packet, its jitter mean equals the packet's arrival time. Because every attack class was captured in its own short window, on the same dates for all nine devices, these features identify the class by the clock. In a sample of 438,955 records, at least one of the ten features exceeded 10⁶ in every attack record but in only 28.9 per cent of benign records. A LightGBM classifier separated the Gafgyt TCP and UDP floods with 99.8 per cent accuracy using the ten features alone and with 50.5 per cent, chance level, without them. Keeping the features raised eleven-class macro-F1 from 0.879 to 0.963 on a random split and from 0.843 to 0.946 when whole devices were held out, while binary detection was unchanged. The artefact therefore inflates exactly the results that are hardest to obtain honestly: multiclass separation and generalisation to unseen devices. We describe how to detect and remove it, give a realistic multiclass ceiling for N-BaIoT, and offer a short checklist for screening other flow-feature datasets for similar leakage.

REFERENCES

[1]. Alkahtani, H., & Aldhyani, T. H. H. (2021). Botnet attack detection by using CNN-LSTM model for Internet of Things applications. Security and Communication Networks, 2021, Article 3806459. doi:10.1155/2021/3806459

[2]. Arp, D., Quiring, E., Pendlebury, F., Warnecke, A., Pierazzi, F., Wressnegger, C., . . . Rieck, K. (2022). Dos and don'ts of machine learning in computer security. In Proceedings of the 31st USENIX Security Symposium (pp. 3971–3988). Boston, MA: USENIX Association.

[3]. Engelen, G., Rimmer, V., & Joosen, W. (2021). Troubleshooting an intrusion detection dataset: The CICIDS2017 case study. In 2021 IEEE Security and Privacy Workshops (SPW) (pp. 7–12). San Francisco, CA: IEEE.

[4]. Kapoor, S., & Narayanan, A. (2023). Leakage and the reproducibility crisis in machine-learning-based science. Patterns, 4(9), 100804. doi:10.1016/j.patter.2023.100804

[5]. Meidan, Y., Bohadana, M., Mathov, Y., Mirsky, Y., Breitenbacher, D., & Shabtai, A. (2018a). Detection of IoT botnet attacks N_BaIoT [Data set]. UCI Machine Learning Repository. doi:10.24432/C5RC8J

[6]. Meidan, Y., Bohadana, M., Mathov, Y., Mirsky, Y., Shabtai, A., Breitenbacher, D., & Elovici, Y. (2018b). N-BaIoT— Network-based detection of IoT botnet attacks using deep autoencoders. IEEE Pervasive Computing, 17(3), 12–22. doi:10.1109/MPRV.2018.03367731

[7]. Mirsky, Y. (2018). Kitsune-py [Computer software]. Retrieved from https://github.com/ymirsky/Kitsune-py

[8]. Mirsky, Y., Doitshman, T., Elovici, Y., & Shabtai, A. (2018). Kitsune: An ensemble of autoencoders for online network intrusion detection. In Proceedings of the Network and Distributed System Security Symposium (NDSS 2018). San Diego, CA: Internet Society. doi:10.14722/ndss.2018.23204

[9]. Rey, V., Sánchez Sánchez, P. M., Huertas Celdrán, A., & Bovet, G. (2022). Federated learning for malware detection in IoT devices. Computer Networks, 204, 108693. doi:10.1016/j.comnet.2021.108693

[10]. Sarhan, M., Layeghy, S., & Portmann, M. (2022). Towards a standard feature set for network intrusion detection system datasets. Mobile Networks and Applications, 27, 357–370.

[11]. Sasi, T., Lashkari, A. H., Lu, R., Xiong, P., & Iqbal, S. (2025). An efficient self attention-based 1D-CNN-LSTM network for IoT attack detection and identification using network traffic. Journal of Information and Intelligence, 3(5), 375–400. doi:10.1016/j.jiixd.2024.09.001

[12]. Soe, Y. N., Feng, Y., Santosa, P. I., Hartanto, R., & Sakurai, K. (2020). Machine learning-based IoT-botnet attack detection with sequential architecture. Sensors, 20(16), 4372. doi:10.3390/s201643