A Lightweight Double-Stage Scheme to Identify Malicious DNS

Version	Summary	Created by	Modification	Content Size	Created at	Operation
1		Ammar Odeh	--	2789	2023-11-30 10:19:28	\|
2	layout	Camila Xu	Meta information modification	2789	2023-12-04 01:07:22	\|

This entry is adapted from the peer-reviewed paper 10.3390/s23073489

The Domain Name System (DNS) protocol essentially translates domain names to IP addresses, enabling browsers to load and utilize Internet resources. Despite its major role, DNS is vulnerable to various security loopholes that attackers have continually abused.

cybersecurity Domain Name System (DNS) DNS over HTTPS (DoH) artificial intelligence

1. Introduction

Every host on the Internet has a unique IP address that allows users to connect and communicate with it. In the early days of the Internet, users could only access a web server using the server’s IP address. For instance, to visit the Google website, a user must type the server’s IP address, 142.250.178.142, instead of www.google.com. Later, in the 1980s, the number of Internet hosts grew to hundreds of thousands. As a result, it became impractical to memorize and maintain the IP address of every single host on this network ^[1]. Paul Mockapetris solved this problem by introducing the Domain Name System (DNS). This name resolution system maps a hostname to its IP address ^[2]. As initially designed, DNS has a hierarchical tree structure consisting of three layers: the root layer, the top-level domain (TLD) layer, and the authoritative layer ^[2]. The name-to-IP mapping process begins when Internet clients such as web browsers initiate a DNS request and send it to the resolver ^[2]. The resolver passes the request through different servers to look up the corresponding IP address and send it back to the client, as shown in Figure 1.

Figure 1. DNS Name Resolution.

By design, DNS traffic is unencrypted. Such plaintext communication allows attackers to launch attacks on the transmitted DNS packets ^[3]. According to the IDC DNS threat survey report, 88% of the organizations have experienced a DNS-based attack during 2022, with an average of seven attacks per organization. The report also revealed that DNS phishing, DNS hijacking/spoofing, DNS-based malware, and DoS/DDoS attacks were the most common DNS-based attacks. As a result, organizations lost an average of $942,000 per attack ^[4].

Encrypting DNS traffic prevents malicious attackers from intercepting the communication between the end user and the DNS resolver. To improve DNS security, researchers have proposed two protocols: DNS over TLS and DNS over HTTPS, DoT, and DoH, respectively ^[5]^[6]. DNS over TLS (DoT) is a security protocol that embeds the DNS request and response into the standard Transport Layer Security packet (TLS). Using DoT, a web client initiates a TLS session with the resolver, verifies its public key certificates, and calculates the secret key. Once the session is established, the encrypted DNS traffic is exchanged between both parties over a dedicated port (853) ^[5].

Similarly, DNS over HTTPS (DoH) encrypts DNS traffic to preserve the confidentiality and integrity of DNS communications. Unlike DoT, DoH, which is more recent, does not pass DNS data with TLS traffic but through HTTPS messages. These HTTPS messages are sent over port 443 like typical HTTPS traffic ^[6].

2. Background

DNS resolvers convert human-readable domain names into machine-readable IP addresses, serving as the Internet’s equivalent of a phone book ^[7]. DNS requests and replies are typically sent in plaintext (through UDP), implying that anyone able to monitor transmissions, including network administrators and ISPs. Even if a website employs HTTPS, the DNS query itself is unencrypted ^[8]^[9]. Because DNS requests are not encrypted, it is easier for attackers to follow users’ online activities. This lack of privacy significantly affects security and, in some situations, human rights ^[10]. Unencrypted DNS queries can be compared to postcards sent through the mail since anyone who handles the mail might have text printed on the back. As a result, it is not advisable to transmit a postcard containing sensitive or private information ^[11].

Malicious DNS over HTTPS (DoH) can be challenging to identify because it is designed to look like regular DoH traffic ^[12]. Figure 2 shows a few indicators that can suggest malicious activity.

Figure 2. Indicators of malicious activities.

According to the figure, the indicators of malicious activities include:

Sudden spikes in DoH traffic: a sudden increase in network traffic could indicate malicious actors are using DoH to bypass DNS filters ^[13].
DoH traffic to suspicious domains: if a lot of DoH traffic goes to domains known to be associated with malware, phishing, or other types of malicious activity, it could be a sign that malicious actors are using DoH to access those domains ^[14].
Encrypted DoH traffic from known malware-infected hosts: if hosts on the network are known to be infected with malware, and encrypted DoH traffic is coming from those hosts, it could be a sign that the malware is using DoH to communicate with its command and control servers ^[15].
DoH traffic bypassing DNS filters: if the implemented DNS filters block access to known malicious sites and DoH traffic sidesteps these filters, it could be a sign that someone is using DoH to bypass the implemented DNS filters ^[16].

To detect malicious DoH traffic, the system may need to implement specialized tools and techniques, such as deep packet inspection ^[17]^[18], behavioral analysis ^[19]^[20], or machine learning algorithms ^[12]^[21]. Additionally, it is important to stay up-to-date with the latest threats and vulnerabilities related to DoH and to follow best practices for securing your network and endpoints ^[22].

User Datagram Protocol (UDP), an unreliable delivery protocol, was the foundation upon which the Domain Name System (DNS) was first created. At the time, the DNS design’s security met all of the requirements of the Internet. However, the strategy is subject to network protocols for current Internet traffic since it offers name-to-address mapping services for the chain of Internet connectivity ^[23]. Internet-connected networks have developed much more quickly than any other technology globally. The Domain Name System (DNS) has existed on the Internet since its inception and has always been crucial. DNS’ leads users to requested computers, programs, and data by converting domain names to corresponding IPs. Due to the vulnerabilities in DNS system, attackers may use DNS-based malware, DNS amplification, false-positive triggering, DNS tunneling, etc.. To counter these kinds of issues, Google and Cloudflare recently developed and implemented DNS over TLS (DoT) and DNS over HTTPS (DoH) ^[24]. The DoT and DoH standard protocols encrypt DNS traffic between users and DNS resolver servers to provide privacy and security ^[25].

Two protocols, DNS over TLS and DNS over HTTPS, were created to encrypt plaintext DNS communication and shield it from the prying eyes of adversaries, ISPs, and other third parties. Keeping with the analogy, these guidelines seek to enclose all postcards sent through the mail, allowing anyone to send a postcard without being concerned that someone is keeping tabs on what they are doing ^[26]^[27]. Malicious DNS over HTTPS (DoH) is a type of cyberattack that involves using encrypted DNS over HTTPS to bypass traditional network security measures and send DNS queries to a malicious server controlled by an attacker. Machine learning can improve the effectiveness of malicious DoH attacks by creating more sophisticated and evasive attacks ^[19].

Using machine learning, attackers can develop more sophisticated methods of evading detection by security tools, such as using randomized domain names, varying the frequency and volume of DNS traffic, and altering the timing of DNS requests. Machine learning can also identify vulnerabilities in specific network configurations and adapt malicious DoH attacks to take advantage of those vulnerabilities ^[28].

Using machine learning in malicious DoH attacks presents a significant challenge for defenders. It allows attackers to develop more effective and adaptive attack techniques that are more difficult to detect and defend against. However, organizations can protect themselves against malicious DoH attacks by implementing security measures such as monitoring network traffic, implementing threat detection tools that use machine learning, and using encryption and authentication technologies to secure their DNS traffic. Additionally, organizations can invest in training and education programs to ensure their employees know the risks and can identify and report suspicious activity ^[14]^[16].

Malicious actors can use DoH to hide their malicious activities from security tools that rely on traditional DNS protocols for detection. Since DoH encrypts DNS queries, it becomes difficult for security tools to analyze DNS traffic for malicious activity. Malicious actors can use DoH to exfiltrate data from a compromised system. Since DoH encrypts DNS queries, it becomes difficult for security tools to detect and block data exfiltration attempts. Malicious actors can use DoH to bypass security controls that rely on DNS-based blacklists and filtering. Since DoH encrypts DNS queries, it becomes difficult for security tools to block access to malicious domains or IP addresses.

Identifying malicious DNS over HTTPS (DoH) traffic can be challenging, but a hybrid learning method can help. The hybrid learning method combines machine learning algorithms with human expertise to detect and classify malicious traffic accurately. A hybrid learning method can accurately detect and classify malicious DoH traffic by combining machine learning algorithms with human expertise. However, it is important to note that this approach requires significant data and expertise to implement effectively.

3. A Lightweight Double-Stage Scheme to Identify Malicious DNS over HTTPS Traffic Using a Hybrid Learning Approach

Authors in ^[12] evaluated five standard ML models, including K-nearest neighbors (KNN), C4.5 Decision tree (DT), Random Forest (RF), and Naive Bayes (NB), to work on detecting DoH traffic and analyzing the information obtained from the properties of the protected HTTPS connections. A CIRA-CIC-DoHBrw-2020 dataset, Naive Bayes (NB), and an Adaboost Decision tree were all used. As it could detect DoH traffic using RF classifiers with an accuracy of roughly one (99.99%), the findings of ML approaches were encouraging.

Authors in ^[16] employed five classes from the CIRA-CIC-DoHBrw-2020 massive cybersecurity data collection from the UNB site as the subject of the study: RNN, RFC, DTC, LSTM, and GRU. They also used GBC, KNC, and XGBoost. Accuracy, MAE, MSE, classification tables, and confusion matrices were additional evaluation metrics.

The results show that for the malicious DoH traffic, XGBC and RFC have achieved the highest accuracy and F1-score. The accuracy rate for the XGBC model was 99.22%. Moreover, RFC had an accuracy of 99.11%. Moreover, the GBC algorithm’s accuracy was 99%. The XGBC, RFC, and GBC models obtained the lowest MAE values of 1.13%, 1.22%, and 1.45%, respectively. Regarding MSE, the GRU model had the lowest error at 1.26%, followed by the RNN model at 1.28%, the LSTM model at 1.28%, and the XGBC model at 1.83%. Also, compared to the findings of the other models, the XGBC model correctly identified 24,793 samples, the largest number. Just behind XGBC, 24,787 samples accurately identified for all four classes came in second place.

The studies show that the XGBC and RFC classifiers perform the best in this data collection. Although the authors of ^[16] aimed to categorize different DNS tunneling technologies to launch malicious DoH traffic, the data’s lack of diversity poses a con-straint. Also, new methods of data theft are being created quickly. As a result, it is essential to account for the wide range of tunneling assaults. Further research is required to examine the proper procedures to make these tunnels safe and to employ a large-scale data set with as many potential tunnels as possible.

The authors of ^[21] reduced unnecessary noise, implemented feature selection techniques, and offered explainable features, demonstrating more accurate and effec-tive identification of fraudulent DoH traffic. This work aims to advance previous re-search and create a more suitable model for practical application by eliminating noise, applying feature selection techniques, and emphasizing feature explainability. The results show that the light gradient boosting machine (LGBM) produced the highest accuracy-to-training time ratio, reaching 0% error utilizing 20 top features after removing five overfitting features.

The precise identification of DoH is the focus of ^[24] discussion of the potential of encrypted traffic analysis. The objective is to assess whether machine learning can extract any information from HTTPS extended I.P. traffic data. The results of the studies indicate that DoH recognition is accurate to over 99.9%. Also, as the authors have found (using produced datasets) substantial differences in the behavior of Firefox, Chrome, and Cloudflare, it is possible to identify the application used for DoH communication.

Given the relative youth of the DoH protocol, several earlier works on shielding users against rogue connections have been published. Most earlier research focused on identifying DoH connections since, as they have already mentioned, employing this protocol means that current security mechanisms are bypassed, making it the best security practice to recognize and prevent these connections. Authors in ^[25] focused on identifying the dangers of utilizing the DoH protocol, outlined the approaches for spotting DoH traffic, and proposed a neural network approach for seeing DoH traffic. They employed a dataset acquired from edge routers through IPFIX/NETFLOW, and the achieved prediction accuracy was 80% for non-normalized data and more than 95% for cleaned and normalized data.

The authors in ^[29] utilized six machine-learning techniques to propose a system-atic two-layer method for identifying DoH traffic and separating benign from malicious DoH traffic. The effectiveness of the proposed approach was evaluated using accuracy, precision, recall, F-score, confusion matrices, ROC curves, and the significance of the features. The findings demonstrated that the LGBM and XGBoost algorithms outperform the competition in practically all classification parameters, achieving a maximum accuracy of 100% in the layer one and layer two classification tests. Out of four-thousand test datasets, the LGBM algorithm correctly identified one DoH traffic test as a non-DoH test. Out of the 34 features taken from the CIRA-CIC-DoHBrw-2020 dataset, it was discovered that source IP is the most important feature for separating DoH traffic from non-DoH traffic in layer one, followed by destination IP. In contrast, only destination IP is a crucial component for LGBM and gradient boosting algorithms to distinguish between benign and malicious DoH traffic at layer two.

The research methods in ^[30] have been modified by removing overfitting features and building a useful model from generic features to make them more amenable to practical uses. Removing overfitting features and irrelevant data allows for a quicker, more thorough investigation of DoH detection. The authors recommend the LGBM model due to its exceptional classification accuracy and short training time compared to other machine learning classifiers, such as Random Forest, Decision tree, and XGBoost. This model can distinguish between non-DoH and DoH data and malicious and benign DoH traffic. These encouraging findings open new directions for DoH classification research, including how deep learning outperforms LGBM in terms of speed and accuracy ^[30].

In ^[31], the authors used a novel machine-learning architecture to construct an explainable AI solution. In particular, they used the publicly accessible CIRA-CIC-DoHBrw-2020 information to develop a precise method for identifying and categorizing DNS over HTTPS assaults. For the given classification challenge, the proposed model achieved extremely high precision, recall, and F-score of 99.91%, 99.92%, and 99.91%, respectively. However, detailed methods and materials should be provided for reproducablity.

In ^[32], collections of DoH resolvers are connected to Firefox over various test sessions. The collected traffic is next examined for DoH traffic using temporal characteristics and packet sizes. The proposed model detected DoH requests from other HTTPS traffic using factors related to packet size. Also, a preliminary step is demonstrated that enables external listeners to identify requested websites based on trends in DoH packet sizes. Last but not least, this research offers recommendations for improving DoH by padding the inquiries to increase the privacy benefits offered by DoH. The results of this study demonstrate that DNS privacy still confronts difficulties and that a complete examination of the dangers to DoH privacy is necessary.

In ^[33], authors explored the classification of DNS over HTTPS communication. The majority of linked works make use of various dataset properties. The trained models cannot be generalized to other network environments since certain incorporated attributes are exclusive to some. A machine-learning model’s generalization is crucial since it will influence how well it performs in different network contexts. To aid in generalizing deep learning models, the authors constructed an adequate data processing pipeline to handle the CIRA-CICDoHBrw-2020 time series dataset, including feature selection and data imbalance handling. In addition, they created generalized deep learning models, such as the LSTM and BiLSTM models, to accurately and quickly categorize DoH traffic. Although both models perform well, the BiLSTM model outperforms the LSTM model in accuracy and computation time.

To identify malicious DoH tunneling and create a fully functional DoH detection system that can be integrated with the security operating system of a corporate network, authors of ^[34] presented a detection system for DoH tunneling attacks based on a transformer model. The suggested system is a simple and effective DoH tunneling detection solution for a business network to successfully detect malicious DoH traffic mixed in with HTTPS traffic. The suggested system offers a distributed collection mechanism that the security operation center (SOC) can use to gather HTTPS data from any network device and analyze it in real-time to look for malicious DoH traffic. Compared to current suggestions, the detecting module with the Transformer model has numerous advantages. A substantially lower amount of labeled data is needed. Even though it is trained using only 25% of the labeled data, it outperforms earlier techniques in terms of accuracy with the same amount of labeled data, up to 99%.

References

Jose, G.-L.; Mary, K.S.; Carol, A.W. Internet Protocol Handbook. In The Domain Name System (DNS) Handbook; DTIC: Fort Belvoir, VA, USA, 1989; Volume 4.
Paul, M. Domain Names—Implementation and Specification; Internet Engineering Task Force; ISI: Marina del Rey, CA, USA, 1987.
Usman Aijaz, N.; Misbahuddin, M.; Raziuddin, S. Survey on DNS-Specific Security Issues and Solution Approaches. In Data Science and Security; Jat, D.S., Shukla, S., Unal, A., Mishra, D.K., Eds.; Lecture Notes in Networks and Systems; Springer: Singapore, 2021; Volume 132, pp. 79–89. ISBN 9789811553080.
Romain, F. DNS Security for Business Continuity and Resilience; IDC: Needham, MA, USA, 2022.
Hu, Z.; Zhu, L.; Heidemann, J.; Mankin, A.; Wessels, D.; Hoffman, P.E. Specification for DNS over Transport Layer Security (TLS); Internet Engineering Task Force: Fremont, CA, USA, 2016.
Hoffman, P.E.; McManus, P. DNS Queries over HTTPS (DoH); Internet Engineering Task Force: Fremont, CA, USA, 2018.
Chang, D.; Chen, J.Q.; Li, Z.; Li, X. Hide and Seek: Revisiting DNS-based User Tracking. In Proceedings of the 2022 IEEE 7th European Symposium on Security and Privacy (EuroS&P), Genoa, Italy, 6–10 June 2022; pp. 188–205.
Park, J.; Khormali, A.; Mohaisen, M.; Mohaisen, A. Where are you taking me? Behavioral analysis of open DNS resolvers. In Proceedings of the 2019 49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), Portland, OR, USA, 24–27 June 2019; pp. 493–504.
Cheng, Y.; Liu, Y.; Li, C.; Zhang, Z.; Li, N.; Du, Y. In-Depth Evaluation of the Impact of National-Level DNS Filtering on DNS Resolvers over Space and Time. Electronics 2022, 11, 1276.
Pavur, J.; Moser, D.; Lenders, V.; Martinovic, I. Secrets in the sky: On privacy and infrastructure security in dvb-s satellite broadband. In Proceedings of the 12th Conference on Security and Privacy in Wireless and Mobile Networks, Miami, FL, USA, 15–17 May 2019; pp. 277–284.
Böttger, T.; Cuadrado, F.; Antichi, G.; Fernandes, E.L.; Tyson, G.; Castro, I.; Uhlig, S. An Empirical Study of the Cost of DNS-over-HTTPS. In Proceedings of the Internet Measurement Conference, Amsterdam, The Netherlands, 21–23 October 2019; pp. 15–21.
Singh, S.K.; Roy, P.K. Detecting malicious dns over https traffic using machine learning. In Proceedings of the 2020 International Conference on Innovation and Intelligence for Informatics, Computing and Technologies (3ICT), Sakheer, Bahrain, 20–21 December 2020; pp. 1–6.
Badhwar, R.; Badhwar, R. Defensive Measures in the Wake of the SolarWinds Fallout. In The CISO’s Transformation: Security Leadership in a High Threat Landscape; Springer: Berlin/Heidelberg, Germany, 2021; pp. 59–64.
Mitsuhashi, R.; Satoh, A.; Jin, Y.; Iida, K.; Shinagawa, T.; Takai, Y. Identifying malicious dns tunnel tools from doh traffic using hierarchical machine learning classification. In Proceedings of the Information Security: 24th International Conference, ISC 2021, Virtual Event, 10–12 November 2021; pp. 238–256.
Lyu, M.; Gharakheili, H.H.; Sivaraman, V. A Survey on DNS Encryption: Current Development, Malware Misuse, and Inference Techniques. ACM Comput. Surv. 2022, 55, 1–28.
S. Alrayes, F.; Maray, M.; Gaddah, A.; Yafoz, A.; Alsini, R.; Alghushairy, O.; Mohsen, H.; Motwakel, A. Modeling of Botnet Detection Using Barnacles Mating Optimizer with Machine Learning Model for Internet of Things Environment. Electronics 2022, 11, 3411.
Rawat, R.; Shedbalkar, K.; Moharir, M.; Deepamala, N.; Kumar, P.R.; Tanmayananda, M. Analysis and detection of malicious activity on doh traffic. In Proceedings of the 2021 2nd Global Conference for Advancement in Technology (GCAT), Bangalore, India, 1–3 October 2021; pp. 1–5.
Parra, G.D.L.T.; Rad, P.; Choo, K.-K.R. Implementation of deep packet inspection in smart grids and industrial Internet of Things: Challenges and opportunities. J. Netw. Comput. Appl. 2019, 135, 32–46.
Naz, N.; Khan, M.A.; Alsuhibany, S.A.; Diyan, M.; Tan, Z.; Khan, M.A.; Ahmad, J. Ensemble learning-based IDS for sensors telemetry data in IoT networks. Math. Biosci. Eng. 2022, 19, 10550–10580.
Fisher, W.W.; Piazza, C.C.; Roane, H.S. Handbook of Applied Behavior Analysis; Guilford Publications: New York, NY, USA, 2021.
Behnke, M.; Briner, N.; Cullen, D.; Schwerdtfeger, K.; Warren, J.; Basnet, R.; Doleck, T. Feature Engineering and Machine Learning Model Comparison for Malicious Activity Detection in the DNS-Over-HTTPS Protocol. IEEE Access 2021, 9, 129902–129916.
Miloslavskaya, N.; Tolstoy, A. Internet of Things: Information security challenges and solutions. Clust. Comput. 2019, 22, 103–119.
Yan, Z.; Lee, J.-H. The road to DNS privacy. Future Gener. Comput. Syst. 2020, 112, 604–611.
Vekshin, D.; Hynek, K.; Cejka, T. Doh insight: Detecting dns over https by machine learning. In Proceedings of the 15th International Conference on Availability, Reliability and Security, Virtual Event, 25–28 August 2020; pp. 1–8.
Deccio, C.; Davis, J. DNS privacy in practice and preparation. In Proceedings of the 15th International Conference on Emerging Networking Experiments and Technologies, Orlando, FL, USA, 9–12 December 2019; pp. 138–143.
Csikor, L.; Singh, H.; Kang, M.S.; Divakaran, D.M. Privacy of DNS-over-HTTPS: Requiem for a Dream? In Proceedings of the 2021 IEEE European Symposium on Security and Privacy (EuroS&P), Virtual Event, 6–10 September 2021; pp. 252–271.
Jerabek, K.; Rysavy, O.; Burgetova, I. Measurement and characterization of DNS over HTTPS traffic. arXiv 2022, arXiv:2204.03975.
Qayyum, A.; Usama, M.; Qadir, J.; Al-Fuqaha, A. Securing Connected & Autonomous Vehicles: Challenges Posed by Adversarial Machine Learning and the Way Forward. IEEE Commun. Surv. Tutor. 2020, 22, 998–1026.
Casanova, L.F.G.; Lin, P.-C. Generalized Classification of DNS over HTTPS Traffic with Deep Learning. In Proceedings of the 2021 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), Tokyo, Japan, 14–17 December 2021; pp. 1903–1907.
Banadaki, Y.M.; Robert, S. Detecting malicious dns over https traffic in domain name system using machine learning classifiers. J. Comput. Sci. Appl. 2020, 8, 46–55.
Zebin, T.; Rezvy, S.; Luo, Y. An Explainable AI-Based Intrusion Detection System for DNS Over HTTPS (DoH) Attacks. IEEE Trans. Inf. Forensics Secur. 2022, 17, 2339–2349.
Bernard, A. Solving Interoperability and Performance Challenges over Heterogeneous IoT Networks: DNS-Based Solutions; Institut Polytechnique de Paris: Palaiseau, France, 2021.
Mitsuhashi, R.; Jin, Y.; Iida, K.; Shinagawa, T.; Takai, Y. Malicious DNS Tunnel Tool Recognition using Persistent DoH Traffic Analysis. IEEE Trans. Netw. Serv. Manag. 2022, 2022, 3215681.
Nguyen, T.A.; Park, M. DoH Tunneling Detection System for Enterprise Network Using Deep Learning Technique. Appl. Sci. 2022, 12, 2416.

© Text is available under the terms and conditions of the Creative Commons Attribution (CC BY) license; additional terms may apply. By using this site, you agree to the Terms and Conditions and Privacy Policy.

Upload a video for this entry

Information

Subjects: Computer Science, Cybernetics

Contributors MDPI registered users' name will be linked to their SciProfiles pages. To register with us, please refer to https://encyclopedia.pub/register :

Qasem Abu Al-Haija

Manar Alohaly

Ammar Odeh

View Times: 254

Update Date: 04 Dec 2023

Table of Contents

Video Upload Options

Confirm

1. Introduction

2. Background

3. A Lightweight Double-Stage Scheme to Identify Malicious DNS over HTTPS Traffic Using a Hybrid Learning Approach

References