Big Data-Enabled Network Traffic Forecasting using Machine Learning and Deep learning Models

Abstract

A real-time, efficient big data prediction network for traffic flow has significant practical applications. Traffic flow data has been growing rapidly over the last few years, and the big data age has arrived. This paper outlines a sturdy network traffic forecasting and intrusion detection framework based on Multilayer Perceptron (MLP) framework on the CSE-CIC-IDS-2018 data. The methodology's initial step is thorough data pretreatment, i.e., deleting missing and duplicate values, selecting features with a model-based approach, encoding categorical values, and normalizing input distributions with Standard Scaler feature scaling. Once the data has been split, the training set that has been processed is used in order to create the MLP-based prediction model that is designed to distinguish benign and attack traffic. With 98.97 accuracy (acc), 99.98 precision (Prec), 98.80 recall (rec), and 99.38 F1-score (F1), the suggested model has exceptional performance and can handle the dataset's extreme class imbalance. The outputs of visualization (class distributions, feature importance, confusion matrix, and learning curves) prove that the model has a high level of stability, generalization, and convergence. A comparative study with several classic and DL algorithms on datasets like CSE-CIC-IDS-2017 and UNSW-NB15 also confirms the excellence of the MLP method in network traffic analysis that can utilize big data.