Anomaly Detection by Network Log Analysis Using Machine Learning

Abstract

In today's world everything is or is transforming into digital. So, internet plays major role in connecting one device to another. As internet aids in ease of use of resources and services, it also is vulnerable. Whenever any server/system is connected to a network or internet, it is prone to attacks. These attacks can steal private data, destroy the network, stop services running on the server and many more. Fortunately, network logs contain all the details about network traffic. By analyzing these network logs, we can detect if there was any abnormality. Although we get all the details in network logs, we can't manually check each line as these logs are quite large in size. So, to detect these anomalies effectively and efficiently Machine Learning is very helpful. In this project we are using UNSW-NB15 dataset. Cyber Range Lab of the "Australian Centre for Cyber Security (ACCS)" created this dataset using "IXIA PerfectStorm tool". which consists a both normal activities and attacks. We are detecting these attacks/anomalies using machine learning algorithms. We train models on different algorithms, but random forest gives the best accuracy in this project.