Modifications or the addition of new features in the applications require retesting the web applications, which adds the problem or concern for software testers to retest the applications. Retesting web services/web applications is a relatively hard job for testers, and it also incurs a high cost. ‘ Regression Testing’ (RT) with its subtypes ‘test case selection’, ‘test case reduction’, and ‘test case prioritization’ have been widely examined in research studies [1]. Test case selection approaches aim at increasing the effectiveness based on their measurement capabilities, including the cost, coverage, and fault detection ^[2][3][4][2,3,4]. Test case prioritization (TCP) approaches let researchers’ reorder test cases to reveal maximum faults based on specific criteria. Test cases with the top defects are tested before the test cases with fewer failures [5]. Proposed RT approaches rely on some criteria, and the ontology of systems is one of these criteria proposed to perform test case selection, reduction, and prioritization.

Ontology refers to basic concepts and their relationship in a research domain [6]. Due to ontology reasoning, the ideas which are harder for their explicit expression are derived. Ontology provides further analysis and assessment knowledge, including the accurate classification of faulty and non-faulty software engineering modules. The field of ontology covers the aspects of methodologies and metrics applied to solve issues in software engineering [7]. OntoClean is one of these methods that covers the use of some properties, including unity, rigidity, identity, and dependency [8]. The OntoClean method, along with meta-properties, is proposed to provide entities’ semantic and logical meaning. In [9], researchers made extensions to define ontology by expressing vocabulary concepts and their use. In this entire document, web applications and web services are interchangeably used.

The motivation behind this ‘systematic literature review’ (SLR) is to investigate the continuous lack of discussion on ontology-derived regression testing in several SLRs [10]. Ontology-derived testing can be considered under the domain of regression testing [11].

Table 1 is the illustration of SLRs in the area of regression testing. The first SLR by Qui et al. [12] includes two papers discussing web services’ semantic behavior. Studies ^[13][14][13,14] did not include papers on ontology-based regression testing. A study by de Souza et al. [15] presented a description of web services ontologies. They include two ontology-driven models, ‘web services modeling ontology’ (WSMO) and ‘web services description language semantics’ (WSDL-S). Therefore, our SLR’s motivation is to help researchers and software practitioners working on the ontologies in regression testing by presenting the state-of-the-art ontology-based RT approaches.

Below are the research questions studied for this SLR (see Table 2 ).

A set of keywords was constructed using the words and phrases from research questions. Search strings relevant to the topic of interest and research questions were formulated ^[17][28]. Researchers used well-known research repositories, including IEEE Xplore, ACM digital library, ScienceDirect, Springer Link, Web of Science, and Wiley & Sons, to search the research articles published between 2007 and 2019. The search strings applied to the digital repositories mentioned above are shown in the following Table 3 .

It was found that when rwesearchers applied “regression testing” OR “test case prioritization” strings, many irrelevant studies to the topic of this SLR were returned. ResearchersWe refined theour search strings using Boolean ‘AND’ and ‘OR’ operators between search keywords to find relevant publications.

The obtained research documents were examined by two independent reviewers (MH and IG). This phase mainly aimed at verifying the research articles, which were potentially eligible and could be used for further analysis. At the screening phase, research documents were screened via title, abstract, and keywords. However, two reviewers (MFP and SRJ) read and reviewed full-length documents resulting from the screening phase at the eligibility phase. Since ontology has been used in software testing, reviewers included papers related to the topic “ontology-based regression testing” with great caution in this systematic review. Therefore, 24 research documents were included in this study.

In the following Table 45 , rwesearchers summarize the studies that focused on ontologies in proposing approaches in regression testing.

Table 45 shows ontology-based studies and authors’ information, problems, proposed techniques, advantages, and limitations of the proposed approaches. The proposed approaches mainly addressed cost, testing in an open platform, modification in evolving web systems, and validation of test cases in nuclear safety systems. This SLR shows the remarkable penetration of ontologies in the prioritization of test cases in web services. Ontology-based regression testing approaches were found effective in saving costs and time compared with random testing approaches. Semantics-based designing of approaches can be further used with the help of ‘artificial intelligence’ to monitor the continuous performance of web services.

A study ^[20][40] proposed a risk assessment approach for the ranking of test cases. Researchers in this study considered risk assessment in the context of users’ input/output and pre/post conditions, processes, and ontology constraints. The risk assessment covers the dissatisfaction of regulatory/legal requirements and relates mainly to a process, work, or project that does not achieve the expected functional behavior. The proposed TCP approach does not involve the reliability of web services. A product risk carefully refers to security and safety behaviors ^[24][48], as well as it also concerns the reliability behavior of the web services. Therefore, web services’ reliability factor can improve regression testing and prioritize test cases. In addition to the challenges mentioned earlier, the failure of critical systems is closely related to the proposed maintenance approach ^[25][49], which cannot detect earlier faults. It is still challenging for researchers to optimize the test case selection and TCP in regression testing.

Before SLR, de Souza et al. [15] identified the research gap that the model-based testing has not dealt with because abstract test cases are trivial to map them to executable and concrete test cases. Compared with the paper ^[26][22], scalability is a focused research problem regarding test brokers because they face re-execution of test cases of a suitable size. The scalability issue remains a commonly identified challenge for both ontology-based RT and collaborative testing of web services. Due to web services’ evolution, scalability is still an open issue in the software testing domain. Validation of proposed RT approaches is mentioned as a problem ^[27][50] because all proposed simulation-based approaches do not address the testing of web applications in real-world scenarios. Therefore, the validation problem is close to our identified problem. A study ^[28][19] stated that cloud testing faces the scalability challenge because distributed systems need enhancement in their performance over time. This performance enhancement can combat the increasing workload on web services. Finally, inadequate validation of test cases is a unique challenge for automated test case generation ^[29][51]. Validation of the test cases is crucial for real-time processing to reduce software testing costs and effort.

This SLR identified ontology-based RT approaches proposed between 2007 and 2019. Most of the studies focused on TCG and TCP research topics. Our SLR presented the roadmap by identifying the benchmark approaches along with their objectives. This study identified six challenges of ontology-based RT approaches. The cost and validation were difficulties widely examined in the selected studies. Scalability, risk assessment, failure of critical systems, and reliability are still open regression testing issues. Our SLR presented the effectiveness of ontology-based RT approaches for addressing the failure in systems other than bioinformatics. This SLR’s findings suggested developing more specific simulation tools regarding the security of the critical system in future works.