Machine Learning-Based Anomaly Detection in NFV

Machine Learning-Based Anomaly Detection in NFV: History

View Latest Version

Please note this is an old version of this entry, which may differ significantly from the current revision.

Subjects: Computer Science, Artificial Intelligence

Contributor:

Sehar Zehra

Ummay Faseeha

Wamda Nagmeldin

Network function virtualization (NFV) is a rapidly growing technology that enables the virtualization of traditional network hardware components, offering benefits such as cost reduction, increased flexibility, and efficient resource utilization. Moreover, NFV plays a crucial role in sensor and IoT networks by ensuring optimal resource usage and effective network management.

network function virtualization (NFV)
Internet of Things (IoT)
security challenges

1. Taxonomy of Network-Based Anomaly Detection in NFV

1.1. NFV Security Issues

NFV works in a virtualized environment that has various vulnerabilities. researchers categorized it into two main types, performance-related vulnerabilities and security-related vulnerabilities [33]. Performance-related vulnerabilities occur due to weakness in the network architecture, lack of data flow control and backup devices, the poor configuration of software and security devices, etc. [34] which will affect the performance of the NFV network, and attackers will exploit these vulnerabilities for attacks [35]. Security-related threats, including malicious attacks, are more easily encountered in NFV because NFV is a shared resource architecture, primarily when implemented on a cloud platform. In addition to third-party interference, the use of public networks for communication also makes NFV security more vulnerable than traditional hardware networks [36].

1.2. Network-Based Anomaly Technique

Anomaly detection techniques are used to identify the abnormal behavior of the overall network and identify not only active and passive attacks but also dynamic and novel malicious attacks [37]. Anomaly detection techniques have some advantages over firewalls or other malware tools, as they can identify abnormal behavior across hosts, networks, and distributed levels.

Network anomaly detection involves monitoring traffic, analyzing various metrics, and using techniques such as statistical analysis, machine learning, and rule-based methods to detect anomalous behavior [39]. In an NFV network, network functions can be dynamically deployed, scaled, and migrated, making it difficult to detect anomalies. Therefore, specialized techniques and tools are needed, such as distributed monitoring and analysis, and techniques that focus on detecting anomalies in the behavior of virtualized network functions themselves [40]. Network anomaly detection in NFV is a specialized form of anomaly detection that focuses on identifying anomalies within virtualized network functions in an NFV environment [41].

1.2.1. Approaches for Anomaly Detection

Anomaly detection has different approaches to finding anomalies in the network, but three of them are more commonly implemented, that is, statistical-based, knowledge-based, and machine learning-based approaches [42]. In statistical-based anomaly detection, abnormalities related to network data traffic are identified using statistical measures, e.g., [43] the mean, standard deviation, uni-variant, and multi-variant. There are several efficient statistical methods for analyzing the anomaly’s existence, such as an operational model, Markov model, outlier model, clustering model, multivariate model, and time series model, etc. [44]. Knowledge-based anomaly detection uses a set of rules to identify malicious behavior; these rules are defined based on suspicious behavior observed from past knowledge of adverse attacks [45]. Therefore, it is also known as a rule-based anomaly detection technique. Machine learning-based anomaly detection uses the automatic approach of classifying normal and abnormal data with the help of a data mining approach [46].

1.2.2. Classification of Anomaly Detection

Machine learning-based anomaly detection is classified into three main approaches, supervised, semi-supervised, and unsupervised anomaly detection. In recent research, a combination of these approaches is used in anomaly detection for NFV networks [47]. Researchers have proposed a method using semi-supervised learning to identify network anomalies and then using supervised learning to classify them as benign or malicious. Others have proposed using unsupervised learning for anomaly detection and then applying semi-supervised learning to identify the root cause of the anomaly. These approaches have shown promising results in detecting and mitigating anomalous behavior in NFV networks [48].

a.: Supervised Model

In supervised anomaly detection, researchers create a model that works on a trained dataset and categorizes the data into two labels, i.e., normal and abnormal [49]. The system collects information regarding the network and compares it to the labeled data; if the data record is more likely to be routine data, then it is considered normal, while on the other hand, if the data are more likely to be abnormal, then it is considered to be an anomaly [50].

b.: Semi-supervised Model

The supervised model depends on the labeled dataset; therefore, the labeled dataset should be of good quality [51]. A semi-supervised model works only on a single label, i.e., a normal dataset; in this approach, if the collected data do not match the normal dataset, then it is considered an anomaly, but this approach does not identify all types of anomalies [52].

c.: Unsupervised Model

Unsupervised is an efficient but complicated approach to finding an anomaly in the network. It does not use any label dataset; it works on instance data and efficiently identifies novel anomalies in the network [53]. The unsupervised approach uses raw measurements and data related to normal behavior to help the system identify novel and dynamic anomalies. Therefore, it is also known as a behavior-based model. There are several unsupervised techniques, such as adaptive threshold-based, clustering, Bayesian belief networks, and principal component analysis [54].

1.2.3. Causes of Network Anomaly Detection

Anomaly detection identifies security vulnerabilities by finding anomalies in the system’s normal behavior [55]. There are several causes of network anomalies, such as network component failure, non-control network traffic, improper monitoring, improper security perimeters, flash crowd, etc. [56].

a.: Network Component Failure

The network component includes hardware- and software-related components, such as routers, firewalls, VNFs services functions, etc. If these components fail during critical data communication, it causes an anomaly, a performance-related issue [57].

b.: Non-Control Network Traffic

Non-control network traffic is a serious issue that causes the network to be unavailable; an attacker exploits this vulnerability and makes the victim server unavailable for legitimate users, which causes anomalies in the network traffic behavior [58].

c.: Improper Monitoring

The access and login of an unauthorized user, weak security monitoring, avoiding unnoticed events, and interruption in the network all come under improper monitoring and cause anomalies in the network [59].

d.: Improper Security Perimeters

Security perimeters include the security measures taken by the network administrator, and these perimeters also cause anomalies in the network. If security perimeters are not strong enough, the network will easily be compromised [60]. The attacker always tries to take advantage of such security vulnerabilities.

e.: Flash Crowd

The flash crowd is also one of the causes of an anomaly in the NFV [61]. Flash crowd means the network is overloaded with legitimate traffic, and many legal users try to access the server, creating abnormal network traffic [62].

1.2.4. Use Cases of Anomaly Detection

Anomaly detection is used in various scenarios in NFV networks to identify performance-related and security-related issues [63]. Some popular use cases of anomaly detection are intrusion detection, fraud detection, malware detection, data loss prevention, log anomaly detection, etc. [64]. The anomaly detection identifies anomalies in the NFV network in an automated way and generates alerts that help the network to take immediate countermeasures.

1.2.5. Challenges of Anomaly Detection

Anomaly detection is a helpful technique for identifying unusual behavior, through which researchers detect existing and novel intruders or malicious attacks, and it also helps improve the performance of the NFV network [65]. Despite all these, there are several limitations of the anomaly detection technique, a few of which are discussed here.

(a): Runtime Anomaly Detection

Fast and reliable communication has always been the goal of NFV; researchers always want a solution that helps to identify runtime anomalies accurately by inspecting the data traffic without disturbing the legitimate traffic [66].

(b): Reducing False Alarm

The differentiation between normal and abnormal behavior is a difficult task; there is a chance that an anomalous event may occur close to normal behavior, and a normal event may occur close to anomalous behavior [67]. In both cases, researchers have a false alarm. Design such anomaly detection techniques that reduce this false alarm in NFV.

(c): Dimensionality Reduction

The appropriate selection of network traffic features is an important challenge in NFV. Select those network features for anomaly detection to help identify anomalous data traffic without compromising performance [68].

(d): Adaptability to Unknown Attacks

As the communication world grows, new challenges in the form of anomalies exist, which should be dynamically identified by anomaly detection techniques [69].

(e): Infrastructure Attacks

NFV is a virtual network environment that involves third-party to provide network infrastructure; several vulnerabilities exist in this network environment [70]. Therefore, an efficient anomaly detection system is required for such types of vulnerabilities.

Due to these issues, detecting anomalies in the NFV network is not easy. Researchers proposed several anomaly detection methods to overcome these limitations [48].

2. Review and Comparative Analysis of State-of-the-Art Anomaly Detection in NFV

Recently, the detection of malicious attacks in the NFV network has received considerable attention, and new algorithms for detecting such attacks have been developed that use the anomaly detection technique. Anomaly detection can identify malicious attacks in the overall network, while the firewall detects malicious attacks only in the data that pass through the firewall [71]. Therefore, new algorithms for anomaly detection need to be developed to identify anomalies in the NFV network that should overcome all the limitations as researchers discussed in the previous section.

2.1. State-of-the-Art Anomaly Detection in NFV

2.1.1. Anomaly Detection Using SMNRT

Derstepanians, Arman, et al. (2022) [66] proposed a machine learning-based approach for detecting anomalies in network function virtualization (NFV) infrastructures. The proposed method, simple median near real-time (SMNRT), is a hybrid approach that combines unsupervised and supervised learning techniques. The unsupervised part of the system uses a clustering algorithm to group similar data points into clusters, with anomalous data points identified as outliers. The supervised part trains a machine learning model to classify data points as either normal or anomalous. The proposed system is evaluated on a dataset of network traffic data from a real-world NFV infrastructure and achieves high detection accuracy, with an F1 score of over 0.9. The paper’s methodology involves four main steps, including data pre-processing, feature extraction, unsupervised clustering, and supervised classification. The evaluation of the proposed system includes comparing its detection performance with other state-of-the-art anomaly detection methods, demonstrating its effectiveness in detecting anomalies in near real time with high accuracy.

2.1.2. Matrix Differential Decomposition

Chen, Jing, et al. [72] proposed the matrix differential decomposition (MDD) method of anomaly identification in the NFV network. They designed a technique that works in three phases. In the first phase, a prototype model is implemented in the NFV network that collects and monitors the NFV network traffic, and its behavior is analyzed. The second phase implements the matrix differential decomposition model (MDD) that identifies the anomaly in the NFV network. In the last step, the proposed algorithm is tested experimentally, evaluated on three NFV networks individually, and its outcomes are studied. The MDD algorithm for anomaly detection and localization not only gives good results in identifying multiple anomalies at a single time but also prevents anomalies due to the localization of network devices.

2.1.3. Machine Learning-Base Early Anomaly Detection

Elmajed, Arij, Armen Aghasaryan, and Eric Fabre et al. [73] presented a machine learning-based anomaly detection algorithm focusing on two main challenges to identifying the anomaly in the NFV network: first, to detect faults before they severely affect the network, and secondly, to take countermeasures before the unavailability of NFV services. For this purpose, an experimental cloud-based NFV application was created that is isolated from all other applications, and this environment contains few virtualized network functions. The authors injected a series of resource perturbations and collected multiple metrics of the system behavior. In the next step, using different machine language approaches, they identified the anomaly in the system. They studied four machine learning (ML) approaches and compared their metrics results; the random forest (RF), XGBOOST, and KNN algorithms gave accuracy above 90%, while the max-likelihood classifier had 84% accuracy. After analyzing the fault localization and identification performance, RF and XGBOOST gave the best results in classifying the different types of anomalies. Despite these results, the proposed model further needs to improve the method of anomalies in the NFV network in a more generalized way.

2.1.4. Tree-Based Anomaly Detection

Girish, L., et al. [74] discussed the isolation forest algorithm for anomaly detection in NFV networks, which is an unsupervised anomaly detection approach. In this method, each occurring event of the data can be efficiently separated and works as a decision tree. The highly sensitive nature of the isolation forest helps to isolate abnormal data points toward the end nodes of the decision tree and normal data points toward the root. The feature that kept anomalies isolated from normal points originally helps to detect abnormalities in the NFV network. The isolation forest algorithm is tested by injecting the anomalies in the NFV network and collecting 12 different metrics’ data. Results show that the isolation tree algorithm efficiently detects anomalies dynamically in the NFV network.

2.1.5. SLA-Aware Anomaly Detection

Hong, Jibum, et al. [75] proposed a machine learning algorithm for anomaly identification in NFV networks using service level agreement (SLA) violation and some of the VNF performance features. The SLA-Aware algorithm work in three steps. The first step is virtual network orchestration, in which a monitoring function operates on NFVI (NFV infrastructure) and collects data on different VNFs in the network in terms of metrics. The second preprocessing step converts the collected information into valid training models and analyzes the data regarding anomaly detection. They divided the data into two categories; normal and abnormal data. The anomalous data are further categorized based on VNF performance and SLA violations. VNF performance includes data that identify packet drops due to the unavailability of VNF resources. SLA violations contain data representing the time that the service does not respond to the request. The last step is training models; in this step, among several machine learning-supervised anomaly detection algorithms, they selected the four best models based on testing. The chosen algorithms are distributed Ran.F (random forest), Gradient Bo.M (boosting machine), Extreme G_B (X gradient boost), and Feed_forward NN (F neural network). The Gradient Bo.M algorithm performs best among these four top algorithms. The results show that the implemented architecture of 95% accurately identified anomalies in the NFV network.

2.1.6. Markov Chain and K-Means Method

Blaise, Agathe, Stan Wong, and A. Hamid Aghvami, et al. [76] proposed a decision-based machine learning algorithm to identify the anomaly in the NFV network. They analyzed the VNFs service in a forward and backward sequence and found the normal and abnormal patterns of network functions. On detecting any anomaly, an alert is generated and a message is sent to the administrator to isolate the NFV network. The whole method is divided into two parts. The first part analyzes the virtual network function services using the Markov chain algorithm. In contrast, in the second part, the K-mean pattern detection technique is used to distinguish the normal behavior or abnormal behavior of the NFV services. The property of the system’s future state depends on the current state because Markov does not store any information; therefore, it is free from history. researchers apply the properties of the Markov chain function both forward and backward. Every VNF represents the state in terms of two transition metrics that also show their connection. The K-mean creates data clusters and uses them to analyze the network behavior. Since K-Mean works on clustered data, it identifies anomalies more accurately than other algorithms and can produce more accurate results as the cluster size increases.

2.1.7. Distance-Based Anomaly Detection in NFV

In Ref. [77], the proposed framework designs a legitimate behavior model at runtime to monitor the network traffic in an NFV network. When an anomaly is detected, the administrator initiates a mitigation process using the root cause analysis technique. This method uses distance-based clustering techniques to develop a legitimate model for anomaly detection. The proposed method efficiently identifies anomalies with low latency rates and reduces the false alarm rate.

2.1.8. Intelligent Orchestration of NFV for Anomaly

Silva, Fernando, and Alberto Schaeffer-Filho, et al. [78] proposed a method for anomaly detection in NFV using a supervised learning technique. The method was implemented in NFV orchestration and management block to monitor the data traffic. The main objective of this technique is to monitor all the incoming traffic; if any anomaly or malicious traffic is found, the proposed module automatically instantiates the network function, which helps with anomaly mitigation. The proposed method is efficient because it is integrated with the NVF orchestration and management module and minimizes resource usage. The experimental evaluation shows that the proposed method identifies anomalies with more than 90% accuracy in the NFV network.

2.1.9. IFTM-Based Anomaly Detection in NFV

Schmidt, Florian, et al. [79] proposed a model that implemented an unsupervised learning approach for anomaly detection in the NFV network. This method consists of an automatic function for identification and a threshold-based technique for classifying data traffic into normal and abnormal behavior. Due to two main tasks, i.e., identity and threshold values, the proposed method is called IFTM. IFTM identifies anomalies dynamically. The first function monitors the network traffic and identifies its behavior; if the traffic data are found to be abnormal, they are sent to the threshold function, where researchers classify their behavior as malicious or normal. This method gives 98% accurate results and also reduces the false alarm rate. However, this method has some limitations; the IFTM method is an expert system and needs some administrative control to handle it. Therefore, a method should be designed with an automatic system for anomaly detection without the intervention of any administration.

2.1.10. LSTM-Based Anomaly Detection in NFV

Alessio Diamanti, Jose Manuel. S.V., et al. [80] proposed an event-driven unsupervised machine-learning method to detect anomalies in the NFV network. This method provides a fully automated anomaly detection solution and identifies anomalies at runtime. The proposed method works in two steps; in the first step, they designed different software modules for other network functions. They used long short-term memory (LSTM) autoencoders and identified whether the data were nominal or anomalous. In the second phase, if any anomaly was found, it was sent to the root cause analysis module, where the anomaly’s mitigation occurs. The LSTM autoencoder works on the radiography visualization approach. This method identifies anomalies dynamically in a heterogeneous environment. This method gave 90% accurate results, but this method works on the virtual layer. Therefore, the proposed design should be extended to physical and cross-layer anomalies.

2.1.11. Unsupervised Neural Network SOM

Lanciano, Giacomo, et al. [81] presented an approach for detecting anomalies in virtual networks using unsupervised machine learning techniques. The proposed method involves the use of a self-organizing map (SOM), which is an unsupervised neural network that can cluster similar data points together. The SOM is trained on network traffic data to create a map of the normal network behavior. During the detection phase, new traffic data are input into the SOM, and if the data deviate significantly from the normal behavior, an anomaly is detected. The authors evaluated their approach on a simulated virtual network environment and found that it was able to detect various types of anomalies, including denial-of-service (DoS) attacks and port scans, with a high degree of accuracy. Overall, the paper demonstrates that unsupervised machine learning techniques can be effective for detecting anomalies in virtual networks, and the proposed SOM-based approach shows promise for this task.

2.2. Comparative Analysis of State-of-the-Art Anomaly Detection in NFV

All the above-proposed methods for anomaly detection in the NFV have some strengths and weaknesses. The method proposed by Derstepanians, Arman, et al. [66] was useful for telecommunication and infrastructure services. Telecommunication providers are always ready to deploy easily configurable and cost-effective solutions. Derstepanians, Arman, et al. [66] therefore designed a model that is easily implemented within the VNF services module and uses both supervised and unsupervised methods for anomaly detection. Their approach used VM data for anomaly detection. The method proposed by Jing Chen [72] is a matrix decomposition method, in which they use a three-step procedure to detect the anomaly and solve the device localization problem that generates the anomaly. This method not only gives good results but also reduces the presence of anomalies in the NFV network through the localization of devices.

Arij Elmajed [73] proposed a runtime solution for anomaly detection and focused on two main tasks: detecting anomalies before they affect system performance and taking timely countermeasures. Arij Elmajed [73] implemented his method using four different machine learning algorithms and studied their behavior in terms of accuracy. Girish and Dr. Sridhar [74] used the isolation forest algorithm technique to identify anomalies in the NFV network and create a decision tree for data. This decision tree dynamically separates the anomalous data from the norm data and shows good results. Jibum Hong, Suhyun, and Jae Hyoung [75] used service level agreement (SLA) and performance characteristics of VNF. This approach works in three steps: monitoring data traffic, analyzing data, and taking countermeasures to mitigate anomalies. The analyzed phase plays a major role in detecting anomalies. It separate the data into two main categories: anomalies due to VNF performance or SLA violations.

Agathe Blaise [76] proposed a decision-based machine algorithm that detects anomalies in forward and backward sequences and generates an alert message. The method works in two steps. The first uses the Markov chain algorithm to detect anomalies, and the second uses the K-means algorithm to generate an alert message if any anomaly occurs.

Anton Gulenko, Florian Schmidt [79] used a distance-based clustering model to identify anomalies in a NFV network and implement a mitigation process using root cause analysis. This method relies on human interaction to deal with anomalies but has low latency and reduces the false alarm rate. Fernando Silva and Alberto, Schaeffer-Filho [78] proposed a method implemented in the NFV orchestration and management block to identify anomalies in the network without any human interaction automatically. The proposed method is efficient and reduces resource consumption. Florian Schmidt and Anton Gulenko [77] proposed a method that automatically identifies anomalies in the network’s data traffic and, after finding any anomalous data, checks whether the data are malicious or not. For this check, it uses a threshold value. Alessio Diamanti, Jose Manuel [80] proposed a method that identifies anomalies in a heterogeneous environment and provides zero-touch network orchestration in the NFV. This method dynamically and automatically detects anomalies but operates at the virtual layer of the NFV network [82]. Here, researchers compare these methods and analyze their efficiency and effectiveness in identifying anomalies in NFV. From the above-proposed methods, researchers conclude some important facts regarding the anomaly detection technique in the NFV:

Supervised methods identify anomalies in the NFV network more quickly and accurately as compared to unsupervised methods.
Supervised methods are either implemented in NFV orchestration and management block or VNFs services function block; this technique reduces the cost and resource utilization.
Unsupervised methods are complex compared to supervised methods but detect novel anomalies in the NFV network.
Unsupervised methods provide a runtime anomaly detection mechanism and are implemented as separate modules or service functions.
Unsupervised methods have more false alarm rates than supervised methods [83].
Unsupervised methods provide a more generalized solution for anomaly detection than supervised methods.
Supervised methods also provide a mitigation process using root cause analysis and reduce costs by integrating with the NFV infrastructure.
Unsupervised methods provide a zero-touch network [80] monitoring environment and automatic anomaly detection approach in the NFV, whereas supervised methods need human interaction to handle anomalies.
Unsupervised methods also work in heterogeneous data environments in runtime scenarios [79].

researchers also study other surveyed papers on state-of-the-art anomaly detection in the NFV network. For instance, Wang, Song, et al. [22] (2021) discuss anomaly detection in network security, including various machine-learning techniques that can be used for anomaly detection. However, they do not specifically focus on anomaly detection in NFV-based networks. Nonetheless, the paper provides a good overview of the techniques that can be used for network anomaly detection, which could be useful in the context of NFV-based networks as well [84].

2.3. Quantitative Comparison of State-of-the-Art Anomaly Detection in NFV Network

Accuracy measures how often a model correctly predicts the outcome. Precision measures how often the model is correct when it predicts a positive outcome. It is like asking “How many of the positive predictions were correct?” Precision is important when researchers want to avoid false alarms. Recall measures how often the model correctly predicts a positive outcome out of all the true positive outcomes. Recall is important when researchers want to ensure that researchers do not miss any positive outcomes. The F-measure is a harmonic mean of precision and recall. The F-measure is useful when researchers want to balance the trade-off between precision and recall.

Through the comparison of accuracy, recall, precision, and F1-score across different anomaly detection methods, researchers can determine which method is the most effective in detecting anomalies in the NFV network. It is essential to focus on these quantitative features to ensure that the selected method can accurately and effectively identify anomalies while minimizing false positives. The metrics are presented in a tabular form for each proposed method, except for DBCAD and IFTM, which are marked as “NSP” (not specified) due to the lack of reported results in the corresponding paper. The table aims to provide a quick comparison of the performance of different proposed methods in terms of anomaly detection accuracy. The accuracy metric indicates the percentage of correctly classified instances among all instances. According to this metric, SOM [81] shows the best performance among all methods, achieving an accuracy of 0.9959, followed by SMNRT [66] and SYRROCA [80] with accuracies of 0.981 and 0.974, respectively.

Precision indicates the proportion of true positives to the total number of positive predictions. Among all methods, SOM [81] shows the best precision with a score of 0.9803, followed by MCKM [76] with a precision score of 0.95.

Recall indicates the proportion of true positives to the total number of actual positive instances. Among all methods, SOM [81] shows the highest recall score with a value of 0.9992, indicating that it can identify almost all positive instances as anomalies.

Among all methods, SOM [81] also shows the highest F1-score with a value of 0.9896, followed by SMNRT [66] and SYRROCA [80] with F1-scores of 0.976 and 0.94, respectively.

Overall, the results suggest that SOM is the most effective method for anomaly detection in NFV networks based on the considered performance metrics. However, it is important to consider other factors, such as complexity, scalability, and robustness, when choosing an appropriate anomaly detection method for a specific NFV environment.

This entry is adapted from the peer-reviewed paper 10.3390/s23115340

© Text is available under the terms and conditions of the Creative Commons Attribution (CC BY) license; additional terms may apply. By using this site, you agree to the Terms and Conditions and Privacy Policy.