Search

The high cost of the test can be dramatically reduced, provided that the coverability as an inherent feature of the code under test is predictable. This article offers a machine learning model to predict the extent to which the test could cover a class in terms of a new metric called Coverageability. The prediction model consists of an ensemble of four regression models. The learning samples consist of feature vectors, where features are source code metrics computed for a class. The samples are labeled by the Coverageability values computed for their corresponding classes. We offer a mathematical model to evaluate test effectiveness in terms of size and coverage of the test suite generated automatically for each class. We extend the size of the feature space by introducing a new approach to define submetrics in terms of existing source code metrics. Using feature importance analysis on the learned prediction models, we sort sources code metrics in the order of their impact on the test effectiveness. As a result of which we found the class strict cyclomatic complexity as the most influential source code metric. Our experiments with our prediction models on a large corpus of Java projects containing about 23,000 classes demonstrate the Mean Absolute Error (MAE) of 0.032, Mean‐Squared Error (MSE) of 0.004, and an R2 score of 0.855. Compared with the state‐of‐the‐art coverage prediction models, our models improve MAE, MSE, and an R2 score by 5.78%, 2.84%, and 20.71%, respectively. [ABSTRACT FROM AUTHOR]

• Testability-driven development (TsDD) approach to developing software is introduced. • TsDD complements TDD with a cycle of predicting and refactoring for testability. • A hybrid machine learning model is proposed for efficient testability evaluation. • The impact of different refactorings on software testability is discussed. • The TsDD practice improves class testability by 77.81 % while reducing testing time. Test-first development (TFD) is a software development approach involving automated tests before writing the actual code. TFD offers many benefits, such as improving code quality, reducing debugging time, and enabling easier refactoring. However, TFD also poses challenges and limitations, requiring more effort and time to write and maintain test cases, especially for large and complex projects. Refactoring for testability is improving the internal structure of source code to make it easier to test. Refactoring for testability can reduce the cost and complexity of software testing and speed up the test-first life cycle. However, measuring testability is a vital step before refactoring for testability, as it provides a baseline for evaluating the current state of the software and identifying the areas that need improvement. This paper proposes a mathematical model for calculating class testability based on test effectiveness and effort and a machine-learning regression model that predicts testability using source code metrics. It also introduces a testability-driven development (TsDD) method that conducts the TFD process toward developing testable code. TsDD focuses on improving testability and reducing testing costs by measuring testability frequently and refactoring to increase testability without running the program. Our testability prediction model has a mean squared error of 0.0311 and an R2 score of 0.6285. We illustrate the usefulness of TsDD by applying it to 50 Java classes from three open-source projects. TsDD achieves an average of 77.81 % improvement in the testability of these classes. Experts' manual evaluation confirms the potential of TsDD in accelerating the TDD process. [ABSTRACT FROM AUTHOR]

Appropriate test data are a crucial factor to succeed in fuzz testing. Most of the real-world applications, however, accept complex structure inputs containing data surrounded by meta-data which is processed in several stages comprising of the parsing and rendering (execution). The complex structure of some input files makes it difficult to generate efficient test data automatically. The success of deep learning to cope with complex tasks, specifically generative tasks, has motivated us to exploit it in the context of test data generation for complicated structures such as PDF files. In this respect, a neural language model (NLM) based on deep recurrent neural networks (RNNs) is used to learn the structure of complex inputs. To target both the parsing and rendering steps of the software under test (SUT), our approach generates new test data while distinguishing between data and meta-data that significantly improve the input fuzzing. To assess the proposed approach, we have developed a modular file format fuzzer, IUST-DeepFuzz. Our experimental results demonstrate the relatively high coverage of MuPDF code by our proposed fuzzer, IUST-DeepFuzz, in comparison with the state-of-the-art tools such as learn&fuzz, AFL, Augmented-AFL, and random fuzzing. In summary, our experiments with many deep learning models revealed the fact that the simpler the deep learning models applied to generate test data, the higher the code coverage of the software under test will be. [ABSTRACT FROM AUTHOR]

Test data adequacy is a major challenge in software testing literature. The difficulty is to provide sufficient test data to assure the correctness of the program under test. Especially, in the case of latent faults, the fault does not reveal itself unless specific combinations of input values are used to run the program. In this respect, detection of subdomains of the input domain that cover a specific execution path seems promising. A subdomain covers an execution path provided that each test data taken from the subdomain satisfies the path constraint. Dynamic Domain Reduction, or DDR in short, is a very well-known test data generation procedure, targeted at detection of the subdomains of the input domain that satisfy a given path constraint. In this paper, an improved version of DDR called Rapid Dynamic Domain Reduction, or RDDR in short, is introduced. RDDR is intended to explore subdomains of a program input domain, satisfying a given path constraint. For each feasible path, there is a distinct subdomain of the input domain that causes the program to execute the path. Hereby, we introduce a new metric named domain coverage, to qualify input data sets, in terms of the percentage of the subdomain of a feasible path covered by the data set. The main inspiration behind the domain coverage metric is to support test data adequacy. Our empirical results based on some well-known case studies confirms that RDDR significantly outperforms DDR in terms of speed and accuracy. [ABSTRACT FROM AUTHOR]

Traditional test case generation approaches focus on design and implementation models while a large percentage of software errors are caused by the lack of understanding in the early phases. One of the most important models in the early phases of software development is business process model which closely resembles the real world and captures the requirements precisely. The aim of this paper is to present a model-based approach to automatically generate test cases from business process models. We first model business processes and convert them to state graphs. Then, the graphs are traversed and transformed to the input format of the "Spec explorer" tool that generates the test cases. Furthermore, we conduct a study to evaluate the impact of process characterizations on the performance of the proposed method. [ABSTRACT FROM AUTHOR]

Despite the proven applicability of the spectrum-based fault localization (SBFL) methods, their effectiveness may be degraded due to the presence of coincidental correctness, which occurs when faults fail to propagate, i.e., their execution does not result in failures. This article aims at improving SBFL effectiveness by mitigating the effect of coincidentally correct test cases. In this regard, given a test suite in which each test has been classified as failing or passing and each faulty program has a single-bug, we present a program slicing-based technique to identify a set of program entities that directly affect the program output when executed with failing test cases, called failure candidate causes (FCC). We then use FCC set to identify test cases that can be marked as being coincidentally correct. These tests are identified based on two heuristics: the average suspiciousness score of the statements that directly affect the program output and the coverage ratio of those statements. To evaluate our approach, we used several evaluation metrics and conducted extensive experiments on programs containing single and multiple bugs, including both real and seeded faults. The empirical results demonstrate that the proposed heuristics can alleviate the coincidental correctness problem and improve the accuracy of SBFL techniques. [ABSTRACT FROM AUTHOR]

Considering the fact that faults may be revealed as undesired mutual effect of program predicates on each other, a new approach for localizing latent bugs, namely Hierarchy-Debug, is presented in this paper. To analyze the vertical effect of predicates on each other and on program termination status, the predicates are fitted into a logistic lasso model. To support scalability, a hierarchical clustering algorithm is applied to cluster the predicates according to their presence in different executions. Considering each cluster as a pseudo-predicate, a distinct lasso model is built for intermediate levels of the hierarchy. Then, we apply a majority voting technique to score the predicates according to their lasso coefficients at different levels of the hierarchy. The predicates with relatively higher scores are ranked as fault relevant predicates. To provide the context of failure, faulty sub-paths are identified as sequences of fault relevant predicates. The grouping effect of Hierarchy-Debug helps programmers to detect multiple bugs. Four case studies have been designed to evaluate the proposed approach on three well-known test suites, Space, Siemens, and Bash. The evaluations show that Hierarchy-Debug produces more precise results compared with prior fault localization techniques on the subject programs. [ABSTRACT FROM AUTHOR]