Search

eHealth interventions are becoming a part of standard care, with software solutions increasingly created for patients and health care providers. Testing of eHealth software is important to ensure that the software realizes its goals. Software testing, which is comprised of alpha and beta testing, is critical to establish the effectiveness and usability of the software. In this viewpoint, we explore existing practices for testing software in health care settings. We scanned the literature using search terms related to eHealth software testing (eg, "health alpha testing," "eHealth testing," and "health app usability") to identify practices for testing eHealth software. We could not identify a single standard framework for software testing in health care settings; some articles reported frameworks, while others reported none. In addition, some authors misidentified alpha testing as beta testing and vice versa. There were several different objectives (ie, testing for safety, reliability, or usability) and methods of testing (eg, questionnaires, interviews) reported. Implementation of an iterative strategy in testing can introduce flexible and rapid changes when developing eHealth software. Further investigation into the best approach for software testing in health care settings would aid the development of effective and useful eHealth software, particularly for novice eHealth software developers., (©Oyinda Obigbesan, Kristen Graham, Karen M Benzies. Originally published in JMIR Nursing (https://nursing.jmir.org), 19.07.2024.)

In software engineering, testing has long been a research area of software maintenance. Testing is extremely expensive, and there is no guarantee that all defects will be found within a single round of testing. Therefore, fixing defects that are not discovered by a single round of testing is important for reducing the test costs. During the software maintenance process, testing is conducted within the scope of a set of test cases called a test suite. Mutation testing is a method that uses mutants to evaluate whether the test cases of the test suite are appropriate. In this paper, an approach is proposed that uses the mutants of a mutation test to identify defects that are not discovered through a single round of testing. The proposed method simultaneously applies two or more mutants to a single program to define and record the relationships between different lines of code. In turn, these relationships are examined using the defects that were discovered by a single round of testing, and possible defects are recommended from among the recorded candidates. To evaluate the proposed method, a comparative study was conducted using the fault localization method, which is commonly employed in defect prediction, as well as the Defects4J defect prediction dataset, which is widely used in software defect prediction. The results of the evaluation showed that the proposed method achieves a better performance than seven other fault localization methods (Tarantula, Ochiai, Opt2, Barinel, Dstar2, Muse, and Jaccard).

Computational algorithms have become an essential component of research, with great efforts by the scientific community to raise standards on development and distribution of code. Despite these efforts, sustainability and reproducibility are major issues since continued validation through software testing is still not a widely adopted practice. Here, we report seven recommendations that help researchers implement software testing in microbial bioinformatics. We have developed these recommendations based on our experience from a collaborative hackathon organised prior to the American Society for Microbiology Next Generation Sequencing (ASM NGS) 2020 conference. We also present a repository hosting examples and guidelines for testing, available from https://github.com/microbinfie-hackathon2020/CSIS.

Nowadays, REpresentational State Transfer Application Programming Interfaces (REST APIs) are widely used in web applications, hence a plethora of test cases are developed to validate the APIs calls. We propose a solution that automates the generation of test cases for REST APIs based on their specifications. In our approach, apart from the automatic generation of test cases, we provide an option for the user to influence the test case generation process. By adding user interaction, we aim to augment the automatic generation of APIs test cases with human testing expertise and specific context. We use the latest version of OpenAPI 3.x and a wide range of coverage metrics to analyze the functionality and performance of the generated test cases, and non-functional metrics to analyze the performance of the APIs. The experiments proved the effectiveness and practicability of our method.

Software testing of knowledge-based clinical decision support systems is challenging, labor intensive, and expensive; yet, testing is necessary since clinical applications have heightened consequences. Thoughtful test case selection improves testing coverage while minimizing testing burden. ATHENA-CDS is a knowledge-based system that provides guideline-based recommendations for chronic medical conditions. Using the ATHENA-CDS diabetes knowledgebase, we demonstrate a generalizable approach for selecting test cases using rules/ filters to create a set of paths that mimics the system's logic. Test cases are allocated to paths using a proportion heuristic. Using data from the electronic health record, we found 1,086 cases with glycemic control above target goals. We created a total of 48 filters and 50 unique system paths, which were used to allocate 200 test cases. We show that our method generates a comprehensive set of test cases that provides adequate coverage for the testing of a knowledge-based CDS.

Mutation testing is basically a fault-based software testing technique, which has been proposed to measure the fault detection effectiveness of a test suite using programs with simulated faults (namely mutants). However, mutation testing is time consuming and computationally expensive because of the normal use of a large amount of mutants. Thus, reducing the mutants is of great significance. To address this problem, various mutant reduction techniques have been proposed. Among them, the identification of redundant mutants aims at removing mutants whose test results can be inferred by other mutants. This article proposes a redundant mutant identification technique based on selective symbolic execution called ReMuSSE for weak mutation testing. Redundant mutants could be revealed by identifying those with similar program execution state changes within a program block involving mutated statements. An empirical study was conducted using 13 C programs from different application domains with varying sizes. The empirical results showed that ReMuSSE could identify up to 31.4% redundant mutants and consequentially save up to 35.2% time cost of weak mutation testing. The results demonstrated that ReMuSSE could effectively identify redundant mutants and thus could significantly improve the efficiency of weak mutation testing. [ABSTRACT FROM AUTHOR]

Software is widely used in many application domains. The most popular software are used by millions every day. How to accurately predict and assess the reliability of developed software is becoming increasingly important for project managers and developers. Previous studies have primarily used the software reliability growth model (SRGM) to evaluate and predict software reliability, but prediction results cannot be accurate at particular times or in particular situations. One of the main reasons is that simplified assumptions and abstractions are usually made to simplify the problem when developing SRGMs. Selecting an appropriate SRGM should depend on the key characteristics of the software project. In this article, we propose a deep learning-based approach for software reliability prediction and assessment. Specifically, we clearly demonstrate how to derive mathematical expressions from the computational methods of deep learning models and how to determine the correlation between them and the mathematical formula of SRGMs, and then, we use the back-propagation algorithm to obtain the SRGM parameters. Furthermore, we further integrate some deep learning-based SRGMs and also propose a method for the weighted assignment of combinations. Three real open source software failure datasets are used to evaluate the performance of the proposed models compared to selected SRGMs. The experimental results reveal that our proposed deep learning-based models and their combinations perform better than several classical SRGMs. [ABSTRACT FROM AUTHOR]

Model-based testing has been highlighted in the last few decades. Many improvements have been proposed for this testing method. One improvement is the use of extended regular expressions (EREs) for modeling software behavior, and then using the ERE model to generate test paths. To improve the theory of test generation based on the ERE model, this article presents an algebraic system, named transition algebra, by extending Kleene algebra. Eight operations and their corresponding operational properties, including basic and nonbasic operational properties, are designed for transition algebra, and all such properties are proven. Five examples are given to illustrate the use of ERE modeling and algebraic operations. To ensure the automatic generation of test paths from the ERE model, the article proves that any ERE model constructed by transition algebra can be converted to a set of transition sequences. Compared to Kleene algebra with tests, transition algebra has more operations to build a powerful ERE model and more operational properties to process the ERE model for the generation of test paths. [ABSTRACT FROM AUTHOR]

Spectrum-based fault localization (SBFL) techniques can automatically localize software faults. They employ the program spectrum, such as code coverage profile with test verdicts, to rank the program entities based on their code suspiciousness. In the past decades, researchers have proposed many approaches to optimize these techniques; however, the program structure, which can influence their performance, is not taken into consideration in developing and improving these techniques. In this article, we identify and analyze the effect of the program structure on the application of SBFL techniques. We observe that some specific program structures may introduce structure bias to code suspiciousness and negatively influence the output of SBFL techniques. To mitigate these effects and improve the performance of fault localization, we propose Delta4Ts, a structure-aware technique. Delta4Ts references debugging history to alleviate the impact of structure bias in the calculation of code suspiciousness. It reasons from the observable suspicious value towards the desired suspicious value and the impact of structure bias. To evaluate Delta4Ts under practical constraints, we conduct a controlled experiment using nine widely-studied SBFL formulae on 12 C programs and 6 Java programs. The experiment results show that Delta4Ts can significantly improve the accuracy of the studied SBFL formulae by an average of 34.8% on 12 C programs and 30.6% on 6 Java programs, and improve more on subject programs associated with more history versions or having larger code sizes. [ABSTRACT FROM AUTHOR]

Let us review the main elements of strategies, which we identified in the development of high energy physics simulations. Based on exemplary Monte Carlo simulation projects, several conjectures for the general aspects of work on scientific software are presented, including openness to change, robustness against external changes, modular development processes, and the constant evolution of tests. The analyzed projects were developed over more than 30 years, much longer than the typical lifetime of computer industry software. Some general observations are drawn together with quantitative evaluation on how the quality of software, measured mainly by the precision of the simulation, grows together with the complexity of software and tests. We find these observations interesting, even if attempts to generalize may remain to a degree speculative, as every researcher is tuning that to the specific needs of their own application. In the future, this can be validated, adjusted, or extended by the experiences gained in other projects. [ABSTRACT FROM AUTHOR]

A geographic information system (GIS) provides basic location-enabled services for many different applications related to navigation, education, and telecommunications. It is a foundation for analysis and visualization. Testing GIS is critical, but challenging due to the difficulty to assess the correctness of GIS outputs, which is called the test oracle problem of software testing. Metamorphic testing alleviates the problem by constructing metamorphic relations (MRs) among multiple inputs and outputs of the program under test. In this article, a semiautomated metamorphic testing (SAMT) method, based on the formal MR model and an improved adaptive random testing algorithm, was proposed to the GIS. To evaluate the performance of our approach, we conducted a case study on a superficial area calculation program, a typical component of GIS. Six kinds of MR construction methods were suggested for the GIS domain program testing. The experimental results show that SAMT can detect the mutations effectively that could solve the test oracle problem efficiently. More importantly, there is no need to manual participation in the testing process, except for the MR construction. [ABSTRACT FROM AUTHOR]

Copyright of International Dialogue: East-West is the property of International Center for Slavic Education- Sveti Nikole and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Adaptive testing (AT) is a software testing approach that uses a feedback mechanism to enhance test effectiveness. Its testing strategy can be adjusted online by using the testing data collected during the software testing process. However, it requires complex parameter estimation which results in excessive computational overhead that may hinder the applicability of AT. In this paper, we propose an approach called AT based on moment estimation (AT-ME) to address this problem. The proposed approach uses moment estimation to serve as the algorithm of parameter estimation, which reduces the complexity of AT-ME. In addition, a dynamic length for testing action is set to limit the number of decisions without influencing the test effectiveness. The proposed approach has been validated on the Siemens test suite, which includes seven real programs. The experiments show that AT-ME can reduce the computational overhead of AT without compromising overall testing efficiency. Results demonstrate that AT-ME is a feasible and effective AT strategy. [ABSTRACT FROM AUTHOR]

Running and analyzing a large number of tests in an industrial scenario is labor intensive and time consuming. Hence, it is necessary to select a smaller number of tests for cost reduction as well as fault detection. For a type of nonnumeric systems, the linear-order algorithm for adaptive random testing (ART) (LART) technique is proposed by making tests evenly spread in nonnumeric input domains. To further enhance LART in the industrial scenarios where the number of input categories is too large, a new technique called category selection-based ART (CSBART), in which partial categories are selected to calculate tests’ distances to guide LART, is proposed in this article. The fault-coverage effectiveness of CSBART is evaluated via an empirical study on two large scale billing systems with tens of millions of test cases, and the results demonstrate the promising performance of the proposed CSBART. We also find that, after category selection, CSBART can outperform a more complex and widespread n-per cluster sampling technique that uses K-means clustering to certain extents. [ABSTRACT FROM AUTHOR]

Adaptive random testing (ART) was developed as an enhanced version of random testing to increase the effectiveness of detecting failures in programs by spreading the test cases evenly over the input space. However, heavy computation may be incurred. In this paper, three enhanced algorithms for fixed-size-candidate-set ART (FSCS-ART) are proposed based on the $k$ -dimensional tree (KD-tree) structure. The first algorithm Naive-KDFC constructs a KD-tree by splitting the input space with respect to every dimension successively in a round-robin fashion. The second algorithm SemiBal-KDFC improves the balance of the KD-tree by prioritizing the splitting according to the spread in each dimension. In order to control the number of traversed nodes in backtracking, the third algorithm LimBal-KDFC introduces an upper bound for the nodes involved. Simulation and empirical studies have been conducted to investigate the efficiency and effectiveness of the three algorithms. The experimental results show that these algorithms significantly reduce the computation time of the original FSCS-ART for low dimensions and for the case of high dimensions with low failure rates. The efficiency of SemiBal-KDFC is better than that of Naive-KDFC when the dimension is no more than 8, but LimBal-KDFC is the most efficient of all three. Although the limited backtracking leads only to an approximate nearest neighbor in LimBal-KDFC, its failure-detection effectiveness is, in fact, better than FSCS-ART in high-dimensional input spaces and has no significant deterioration in low-dimensional spaces. [ABSTRACT FROM AUTHOR]

Generation of complex heap structures is required for many software testing and verification techniques, which are used to enhance software reliability. It enables them to work for programs dependent on such structures. Generating all the valid structures within a size bound is time-consuming, and it is hard to predict an input size completely generatable within a time bound. This is due to the depth-first traversal of the candidate space. However, an efficient breadth-first traversal requires a compact state encoding to store candidates for exploration in the next level. In this paper, we present present a novel state encoding technique that stores an incomplete representation of state that can be recovered during search. We build upon the Korat algorithm—demonstrated to effectively generate tests and find bugs for many software programs—and present iKorat, an incremental algorithm for breadth-first exploration of the search space. Our encoding enables a more efficient implementation of iterative deepening that avoids redundant work. Standard iterative deepening algorithms allow a breadth-first search when the underlying algorithm is depth-first by repeating the work of earlier iterations. iKorat, however, uses information from smaller sizes to avoid redundant work for larger sizes. It also enables a new technique for parallelizing Korat by communicating candidates using our encoding. piKorat generates structures of larger sizes in parallel as soon as incremental information is available from smaller sizes. Our evaluation shows that iKorat-based iterative deepening is more efficient than Korat and that piKorat naturally extends the technique for parallel generation of complex structures. [ABSTRACT FROM AUTHOR]

A particularly good software testing strategy is to achieve the underlying testing goal while solving the problems of tradeoffs between testing effectiveness and efficiency. To improve the fault detection effectiveness of software testing, the principle of feedback control theory was adopted, which motivated the proposal of dynamic random testing (DRT). The main idea behind DRT is using the testing results to guide the test case selection to increase the selection probabilities of the subdomains with higher fault detection rates. Previous works show that DRT strategy can achieve better effectiveness than random testing strategy and random partition testing strategy, and has significantly lower computational costs than adaptive testing strategy. However, the essential factors that affect the performance of DRT, i.e., adjusting parameters, initial profile, and test case classification have not been thoroughly investigated. Besides, some experimental assumptions are inconsistent with real scenarios. Therefore, this paper gives a series of investigations on DRT with a set of practical subject programs. More specifically, the effectiveness and efficiency of DRT are presented, and the extended experiments on DRT with relevant factors are conducted. The results indicate that the effectiveness of DRT is robust to different initial profiles and affected noticeably by the adjusting parameter settings and test case classification methods. [ABSTRACT FROM AUTHOR]

As one of the most popular software testing techniques, fuzzing can find a variety of weaknesses in a program, such as software bugs and vulnerabilities, by generating numerous test inputs. Due to its effectiveness, fuzzing is regarded as a valuable bug hunting method. In this paper, we present an overview of fuzzing that concentrates on its general process, as well as classifications, followed by detailed discussion of the key obstacles and some state-of-the-art technologies which aim to overcome or mitigate these obstacles. We further investigate and classify several widely used fuzzing tools. Our primary goal is to equip the stakeholder with a better understanding of fuzzing and the potential solutions for improving fuzzing methods in the spectrum of software testing and security. To inspire future research, we also predict some future directions with regard to fuzzing. [ABSTRACT FROM AUTHOR]

Entity reconciliation (ER) aims to combine data from different sources for a unified vision. The management of large volumes of data has given rise to significant challenges to the ER problem due to facts such as data becoming more unstructured, unclean, and incomplete or the existence of many datasets that store information about the same topic. Testing the applications that implement the ER problem is crucial to ensure both the correctness of the reconciliation process and the quality of the reconciled data. This paper presents an approach based on model-driven engineering that allows the creation of test models for the early integration testing of ER applications, contributing in three main aspects: the description of the elements of the proposed framework, the definition of the testing model, and the validation of the proposal through two real-world case studies. This validation verifies that the early integration testing of the ER application is capable of detecting a series of deficiencies, which a priori are not known and that will help to improve the final result that the ER application offers. [ABSTRACT FROM AUTHOR]

Genetic improvement (GI) uses automated search to find improved versions of existing software. We present a comprehensive survey of this nascent field of research with a focus on the core papers in the area published between 1995 and 2015. We identified core publications including empirical studies, 96% of which use evolutionary algorithms (genetic programming in particular). Although we can trace the foundations of GI back to the origins of computer science itself, our analysis reveals a significant upsurge in activity since 2012. GI has resulted in dramatic performance improvements for a diverse set of properties such as execution time, energy and memory consumption, as well as results for fixing and extending existing system functionality. Moreover, we present examples of research work that lies on the boundary between GI and other areas, such as program transformation, approximate computing, and software repair, with the intention of encouraging further exchange of ideas between researchers in these fields. [ABSTRACT FROM AUTHOR]

It is widely held that debugging cyber-physical systems (CPS) is challenging; many strongly held beliefs exist regarding how CPS are currently debugged and tested and the suitability of various techniques. For instance, dissenting opinions exist as to whether formal methods (including static analysis, theorem proving, and model checking) are appropriate in CPS verification and validation. Simulation tools and simulation-based testing are also often considered insufficient for CPS. Many “experts” posit that high-level programming languages (e.g., Java or C#) are not applicable to CPS due to their inability to address (significant) resource constraints at a high level of abstraction. To date, empirical studies investigating these questions have not been done. In this paper, we qualitatively and quantitatively analyze why debugging CPS remains challenging and either dispel or confirm these strongly held beliefs along the way. Specifically, we report on a structured online survey of 25 CPS researchers (10 participants classified themselves as CPS developers), semistructured interviews with nine practitioners across four continents, and a qualitative literature review. We report these results and discuss several implications for research and practice related to CPS. [ABSTRACT FROM PUBLISHER]