Semi Automated Acceptance Testing Using Information Retrieval

No Thumbnail Available



Journal Title

Journal ISSN

Volume Title


Addis Ababa University


Requirements Engineering(RE) is the discipline of eliciting, analyzing, specifying, validating and managing needs and constraints of a software product. It attempts to document the ideas and needs of the product’s stakeholders in the software requirements specification (SRS) document. Later, when the product is developed and delivered, stakeholders perform acceptance testing. Acceptance Testing is the verification conducted to use it that a software product provides expected behaviors, as expressed in requirements. It is a testing technique performed to determine whether or not the developed software has met the stakeholder's requirements specified initially. One of the first tasks in acceptance testing is to know where a particular requirement is implemented in the software system. As the number of requirements increase, identifying where a particular requirement is implemented in before system becomes challenging and time-consuming, especially in environments where there are not so many expertise. This thesis proposes an information retrieval (IR) based approach to address this problem. The approach uses the use cases in the SRS and the feature descriptions in the software manual, as a query and corpus, respectively, to identify where the use cases described in the SRS are implemented in the software system. To evaluate the proposed approach, we conducted an experiment using two datasets collected from two software products. We observe that the recall values are 100% for all queries. This means that our approach efficiently associates uses cases with their implementation of features in the corresponding software. The average value of F-measure is 44% and 67% in dataset one and two respectively. This indicates that, the proposed approach is able to associates uses cases and features implementation. The Reciprocal rank values of all queries is 1, meaning that the most relevant feature are found at first rank. Hence, the software testers would easily identify the features to test a software product.



Acceptance Testing, Information Retrieval, Software Requirement specification, software product, Query, Corpus, document, metric, Lucene, Indexing, searching