Regression testing is a relevant research field focused on ensuring software works correctly after being modified or when new functionalities are added. The widespread use of information technology techniques has enabled the rapid development of applications.
Modifications or the addition of new features in the applications require retesting the web applications, which adds the problem or concern for software testers to retest the applications. Retesting web services/web applications is a relatively hard job for testers, and it also incurs a high cost. ‘ Regression Testing’ (RT) with its subtypes ‘test case selection’, ‘test case reduction’, and ‘test case prioritization’ have been widely examined in research studies . Test case selection approaches aim at increasing the effectiveness based on their measurement capabilities, including the cost, coverage, and fault detection . Test case prioritization (TCP) approaches let researchers’ reorder test cases to reveal maximum faults based on specific criteria. Test cases with the top defects are tested before the test cases with fewer failures . Proposed RT approaches rely on some criteria, and the ontology of systems is one of these criteria proposed to perform test case selection, reduction, and prioritization.
Ontology refers to basic concepts and their relationship in a research domain . Due to ontology reasoning, the ideas which are harder for their explicit expression are derived. Ontology provides further analysis and assessment knowledge, including the accurate classification of faulty and non-faulty software engineering modules. The field of ontology covers the aspects of methodologies and metrics applied to solve issues in software engineering . OntoClean is one of these methods that covers the use of some properties, including unity, rigidity, identity, and dependency . The OntoClean method, along with meta-properties, is proposed to provide entities’ semantic and logical meaning. In , researchers made extensions to define ontology by expressing vocabulary concepts and their use. In this entire document, web applications and web services are interchangeably used.
The motivation behind this ‘systematic literature review’ (SLR) is to investigate the continuous lack of discussion on ontology-derived regression testing in several SLRs . Ontology-derived testing can be considered under the domain of regression testing .
|Sr. No||Research Focus||Ontology-Based Regression Testing||Number of Scholarly Studies||Reference|
|2.||Regression test case prioritization||x||65|||
|3.||Test case prioritization||x||120|||
|4.||Regression testing of web services||√||30|||
|5.||Regression test case selection||x||47|||
|6.||Test case prioritization||x||69|||
|7.||Test Case prioritization of systems||x||90|||
|8.||Testing of semantic web services||√||43|||
|9.||Agent-based test generation||x||115|||
Table 1 is the illustration of SLRs in the area of regression testing. The first SLR by Qui et al.  includes two papers discussing web services’ semantic behavior. Studies  did not include papers on ontology-based regression testing. A study by de Souza et al.  presented a description of web services ontologies. They include two ontology-driven models, ‘web services modeling ontology’ (WSMO) and ‘web services description language semantics’ (WSDL-S). Therefore, our SLR’s motivation is to help researchers and software practitioners working on the ontologies in regression testing by presenting the state-of-the-art ontology-based RT approaches.
Below are the research questions studied for this SLR (see Table 2 ).
|RQ1||What is the roadmap of regression testing?|
|RQ2||What are the state-of-the-art techniques of ontology-based regression testing?|
|RQ3||What are the challenges and limitations of current approaches?|
|RQ4||What are the possible future research directions?|
|RQ5||What are the unique issues of ontology-based regression testing compared to other regression testing approaches?|
A set of keywords was constructed using the words and phrases from research questions. Search strings relevant to the topic of interest and research questions were formulated . Researchers used well-known research repositories, including IEEE Xplore, ACM digital library, ScienceDirect, Springer Link, Web of Science, and Wiley & Sons, to search the research articles published between 2007 and 2019. The search strings applied to the digital repositories mentioned above are shown in the following Table 3 .
|Data Repository Name||Search String|
|IEEE Xplore||(((((“All Metadata”: Ontology) AND “All Metadata”: Regression Testing) OR “All Metadata”: Test Case Prioritization) AND “All Metadata”: Test Case Selection) AND “All Metadata”: Test Case Generation)|
|ACM digital Library||(+Ontology + Regression + Testing + Test + Case + Prioritization + Test + Case + Selection + Test + Case + Generation)|
|ScienceDirect||“Ontology, regression testing, test case prioritization, Test Case Selection, Test Case Generation.”|
|SpringerLink||‘Ontology AND “Regression Testing” AND (Test OR Case OR Prioritization, OR Test OR Case OR Selection, OR Test OR Case OR Generation)’|
|Web of Science||(Ontology) AND TOPIC: (Regression Testing) OR TOPIC: (test case prioritization) OR TOPIC: (Test Case Selection) OR TOPIC: (Test Case Generation)|
|Wiley & Sons||“Ontology” anywhere and “Regression Testing” anywhere and “Test Case Prioritization” anywhere and "Test Case Selection” anywhere and “Test Case Generation”.|
It was found that when researchers applied “regression testing” OR “test case prioritization” strings, many irrelevant studies to the topic of this SLR were returned. Researchers refined the search strings using Boolean ‘AND’ and ‘OR’ operators between search keywords to find relevant publications.
The obtained research documents were examined by two independent reviewers (MH and IG). This phase mainly aimed at verifying the research articles, which were potentially eligible and could be used for further analysis. At the screening phase, research documents were screened via title, abstract, and keywords. However, two reviewers (MFP and SRJ) read and reviewed full-length documents resulting from the screening phase at the eligibility phase. Since ontology has been used in software testing, reviewers included papers related to the topic “ontology-based regression testing” with great caution in this systematic review. Therefore, 24 research documents were included in this study.
In the following Table 4 , researchers summarize the studies that focused on ontologies in proposing approaches in regression testing.
|Askarunisa et al. ||High cost on web services testing||An automated testing framework||Effective prioritization of web services||No cost resources were used in the proposed approach|
|Askarunisa et al. ||An increase in cost due to source code unavailability||Semantic-based protégé tool||Faults detection is better than the traditional approaches||Coverage criteria is not clearly mentioned|
|Bai et al. ||Web services testing in an open platform||A risk-based approach using semantics||Faults detection with a high impact||The reliability of web services is not mentioned|
|Kim et al. ||Handling changes in the evolving system is a difficult task||Regression test selection||Reduces the overall time to rerun test cases||This approach might have missed some necessary test cases|
|Fan et al. ||Validation of tests in an ad hoc fashion||Domain-specific ontology-based approach||Systematic generation of TCG from safety analysis report||The cost to ensure the safety of a web system is not mentioned|
|Tseng et al. ||Uncertain test coverage from random testing||Domain-specific ontology using requirements||Ensures users’ safety, more effective than random testing||Validation of web systems at large-scale systems is not done|
Table 4 shows ontology-based studies and authors’ information, problems, proposed techniques, advantages, and limitations of the proposed approaches. The proposed approaches mainly addressed cost, testing in an open platform, modification in evolving web systems, and validation of test cases in nuclear safety systems. This SLR shows the remarkable penetration of ontologies in the prioritization of test cases in web services. Ontology-based regression testing approaches were found effective in saving costs and time compared with random testing approaches. Semantics-based designing of approaches can be further used with the help of ‘artificial intelligence’ to monitor the continuous performance of web services.
A study  proposed a risk assessment approach for the ranking of test cases. Researchers in this study considered risk assessment in the context of users’ input/output and pre/post conditions, processes, and ontology constraints. The risk assessment covers the dissatisfaction of regulatory/legal requirements and relates mainly to a process, work, or project that does not achieve the expected functional behavior. The proposed TCP approach does not involve the reliability of web services. A product risk carefully refers to security and safety behaviors , as well as it also concerns the reliability behavior of the web services. Therefore, web services’ reliability factor can improve regression testing and prioritize test cases. In addition to the challenges mentioned earlier, the failure of critical systems is closely related to the proposed maintenance approach , which cannot detect earlier faults. It is still challenging for researchers to optimize the test case selection and TCP in regression testing.
Before SLR, de Souza et al.  identified the research gap that the model-based testing has not dealt with because abstract test cases are trivial to map them to executable and concrete test cases. Compared with the paper , scalability is a focused research problem regarding test brokers because they face re-execution of test cases of a suitable size. The scalability issue remains a commonly identified challenge for both ontology-based RT and collaborative testing of web services. Due to web services’ evolution, scalability is still an open issue in the software testing domain. Validation of proposed RT approaches is mentioned as a problem  because all proposed simulation-based approaches do not address the testing of web applications in real-world scenarios. Therefore, the validation problem is close to our identified problem. A study  stated that cloud testing faces the scalability challenge because distributed systems need enhancement in their performance over time. This performance enhancement can combat the increasing workload on web services. Finally, inadequate validation of test cases is a unique challenge for automated test case generation . Validation of the test cases is crucial for real-time processing to reduce software testing costs and effort.
This SLR identified ontology-based RT approaches proposed between 2007 and 2019. Most of the studies focused on TCG and TCP research topics. Our SLR presented the roadmap by identifying the benchmark approaches along with their objectives. This study identified six challenges of ontology-based RT approaches. The cost and validation were difficulties widely examined in the selected studies. Scalability, risk assessment, failure of critical systems, and reliability are still open regression testing issues. Our SLR presented the effectiveness of ontology-based RT approaches for addressing the failure in systems other than bioinformatics. This SLR’s findings suggested developing more specific simulation tools regarding the security of the critical system in future works.