Search

Test Maturity Model integration (TMMi) is a popular model for maturity assessment and capability improvement of software testing practices in industry. Originally inspired by the Capability Maturity Model Integration (CMMI), and managed by the TMMi Foundation, the TMMi specification provides detailed guidelines for assessing and improving testing capabilities of teams and organizations. We present in this paper a status report about TMMi, motivations for and benefits of using TMMi, and how companies have been ranked in each of its process areas.

PurposeIn the current market scenario, software upgrades and updates have proved to be very handy in improving the reliability of the software in its operational phase. Software upgrades help in reinventing working software through major changes, like functionality addition, feature enhancement, structural changes, etc. In software updates, minor changes are undertaken which help in improving software performance by fixing bugs and security issues in the current version of the software. Through the current proposal, the authors wish to highlight the economic benefits of the combined use of upgrade and update service. A cost analysis model has been proposed for the same.Design/methodology/approachThe article discusses a cost analysis model highlighting the distinction between launch time and time to end the testing process. The number of bugs which have to be catered in each release has been determined which also consists of the count of latent bugs of previous version. Convolution theory has been utilized to incorporate the joint role of tester and user in bug detection into the model. The cost incurred in debugging process was determined. An optimization model was designed which considers the reliability and budget constraints while minimizing the total debugging cost. This optimization was used to determine the release time and testing stop time.FindingsThe proposal is backed by real-life software bug dataset consisting of four releases. The model was able to successfully determine the ideal software release time and the testing stop time. An increased profit is generated by releasing the software earlier and continues testing long after its release.Originality/valueThe work contributes positively to the field by providing an effective optimization model, which was able to determine the economic benefit of the combined use of upgrade and update service. The model can be used by management to determine their timelines and cost that will be incurred depending on their product and available resources.

Architectural Technical Debt (ATD) is considered as the most significant type of TD in industrial practice. In this study, we interview 21 software engineers and architects to investigate a specific type of ATD, namely architectural smells (AS). Our goal is to understand the phenomenon of AS better and support practitioners to better manage it and researchers to offer relevant support. The findings of this study provide insights on how practitioners perceive AS and how they introduce them, the maintenance and evolution issues they experienced and associated to the presence of AS, and what practices and tools they adopt to manage AS., Submitted and accepted to IEEE Software special issue on Technical Debt. This is a preprint

This article presents technical debt (TD) impediments, decision factors, enabling practices, and actions diagrams for TD management in agile software projects. By analyzing diagrams, professionals can avoid the pitfalls, and increase their capacity, for better TD management.

There continues to be an increase in enrollments in various computing programs at academic institutions due to many job opportunities available in the information, communication, and technology sectors. This enrollment surge has presented several challenges in many Computer Science (CS), Information Technology (IT), and Software Engineering (SE) programs at universities and colleges. One such challenge is that many instructors in CS/IT/SE programs continue to use learning approaches that are not learner centered and therefore are not adequately preparing students to be proficient in the ever-changing computing industry. To mitigate this challenge, instructors need to use evidence-based pedagogical approaches, e.g., active learning, to improve student learning and engagement in the classroom and equip students with the skills necessary to be lifelong learners. This article presents an approach that combines learning and engagement strategies (LESs) in learning environments using different teaching modalities to improve student learning and engagement. We describe how LESs are integrated into face-to-face (F2F) and online class activities. The LESs currently used are collaborative learning , gamification , problem-based learning , and social interaction . We describe an approach used to quantify each LES used during class activities based on a set of characteristics for LESs and the traditional lecture-style pedagogical approaches. To demonstrate the impact of using LESs in F2F class activities, we report on a study conducted over seven semesters in a software testing class at a large urban minority serving institution. The study uses a posttest-only study design, the scores of two midterm exams, and approximate class times dedicated to each LES and traditional lecture style to quantify their usage in a face-to-face software testing class. The study results showed that increasing the time dedicated to collaborative learning, gamification, and social interaction and decreasing the traditional lecture-style approach resulted in a statistically significant improvement in student learning, as reflected in the exam scores.

The Common Weakness Enumeration (CWE) community publishes an aggregate metric to calculate the ‘Most Dangerous Software Errors.’ However, the used equation highly biases frequency over exploitability and impact. We provide a metric to mitigate this bias and discuss the most significant software weaknesses over the last ten years.

The high cost of the test can be dramatically reduced, provided that the coverability as an inherent feature of the code under test is predictable. This article offers a machine learning model to predict the extent to which the test could cover a class in terms of a new metric called Coverageability. The prediction model consists of an ensemble of four regression models. The learning samples consist of feature vectors, where features are source code metrics computed for a class. The samples are labeled by the Coverageability values computed for their corresponding classes. We offer a mathematical model to evaluate test effectiveness in terms of size and coverage of the test suite generated automatically for each class. We extend the size of the feature space by introducing a new approach to defining sub-metrics in terms of existing source code metrics. Using feature importance analysis on the learned prediction models, we sort source code metrics in the order of their impact on the test effectiveness. As a result of which, we found the class strict cyclomatic complexity as the most influential source code metric. Our experiments with the prediction models on a large corpus of Java projects containing about 23,000 classes demonstrate the Mean Absolute Error (MAE) of 0.032, Mean Squared Error (MSE) of 0.004, and an R2-score of 0.855. Compared with the state-of-the-art coverage prediction models, our models improve MAE, MSE, and an R2-score by 5.78%, 2.84%, and 20.71%, respectively., Comment: 19 pages, 11 figures

Automatic test oracle generation is a bottleneck in realizing full automation of the entire software testing process. This study proposes a new method for automatically generating a test oracle for a new test input on the basis of several historical test cases by using a backpropagation neural network (BPNN) model. The new method is different from existing test oracle techniques. Specifically, our method has two steps. First, the values of variables are collected as training data when several historical test inputs are used to execute the program at different breakpoints. The test oracles (pass or fail) of these test cases are utilized to classify and label the training data. Second, a new test input is used to execute the program at different breakpoints, where the trained BPNN prediction model automatically generates its test oracle on the basis of the collected values of the variables involved. We conduct an experiment to validate our method. In the experiment, 113 faulty versions of seven types of programs are used as experimental objects. Results show that the average prediction accuracy rate of 74,651 test oracles is 95.8%. Although the failed test cases in the training data account for less than 5%, the overall average recall rate (prediction accuracy of test case execution failure) of all programs is 78.9%. Furthermore, the trained BPNN can reveal not only the impact of the values of variables but also the impact of the logical correspondence between variables in test oracle generation.

We briefly review properties of explainable AI proposed by various researchers. We take a structural approach to the problem of explainable AI, examine the feasibility of these aspects and extend them where appropriate. Afterwards, we review combinatorial methods for explainable AI which are based on combinatorial testing-based approaches to fault localization. Last, we view the combinatorial methods for explainable AI through the lens provided by the properties of explainable AI that are elaborated in this work. We pose resulting research questions that need to be answered and point towards possible solutions, which involve a hypothesis about a potential parallel between software testing, human cognition and brain capacity.

This work attempts to fill a gap by exploring the human dimension in particular, by trying to understand the motivation of software professionals for taking up and sustaining their careers as software testers. Towards that goal, four surveys were conducted in four countries - India, Canada, Cuba, and China - to try to understand how professional software engineers perceive and value work-related factors that could influence their motivation to start or move into software testing careers. From our sample of 220 software professionals, we observed that very few were keen to take up testing careers. Some aspects of software testing, such as the potential for learning opportunities and the importance of the job, appear to be common motivators across the four countries, whereas the treatment of testers as second-class citizens and the complexity of the job appeared to be the most prominent de-motivators., 6 pages. arXiv admin note: substantial text overlap with arXiv:2103.06343, arXiv:1906.06144

To learn about novel software engineering (SE) trends, where do you refer to? In order to document and disseminate their experience / knowledge, many SE practitioners prepare technical materials and share them online as blog posts, white papers and videos. Such materials are often called “grey literature” because they are not formally peer reviewed. By contrast, SE researchers write technical papers that are peer-reviewed and published as academic literature. We observe that, in general, these two communities mostly read literature that is only written by and published within their respective communities. This situation has led to a form of “knowledge divide” between the two communities that, we believe, hurts both communities. By characterizing and contrasting the two types of literature, grey and academic, we discuss how each literature can complement the other and can lead to a richer and more integrated knowledge exchange and dissemination in SE.

This paper explains the basic concepts of GraphQL specification. Types and strategies of software testing according to this concept are given. The application that demonstrates it is implemented. The tests specific for systems that use GraphQL are written and explained.

Although extensive research has been conducted on the characteristics of the agile developer, little attention has been given to the features of the software-testing role. This paper explores the human factors of the software testers working in agile projects through a qualitative study focusing on how these factors are perceived. We interviewed 22 agile software practitioners working in three international companies: 14 testers, five developers, and three designers. Additionally, we observed 11 meetings and daily work of 13 participants in one of the companies. Our findings show that the views on the human factors shaping the agile software tester’s role were crystallized into seven traits, which the agile team members saw as central for the software-testing role: the ability to see the whole picture, good communication skills, detail-orientation, structuredness, creativeness, curiosity, and adaptability. The testers spent half their day communicating and learned how to mitigate the fact that they had to bring bad news to other project members. They also facilitated communication between the business side and development. Based on our results, we propose the seven traits as dimensions to consider for organizations recruiting agile software testers, as well as a reference for IT and non-IT professionals considering a software-testing career.

Testing is broadly classified into three levels: Unit Testing, Addition Testing, and System Testing. Whenever we think of developing any software we always concentrate on making the software bug free and most reliable. At this point of time Testing is used to make the software a bug free. Software Testing has been measured as the most important stage of the software development life cycle. Around 60% of resources and money are cast-off for the testing of software. Testing can be manual or automated. Software testing is an activity that emphases at assessing the competence of a program and commands that it truly meets the excellence results. There are many test cases that help in detecting the bugs so, in this paper we describe about the most commonly used test cases and testing techniques for the error detection.

Classifying test executions automatically as pass or fail remains a key challenge in software testing and is referred to as the test oracle problem. It is being attempted to solve this problem with supervised learning over test execution traces. A programme is instrumented to gather execution traces as sequences of method invocations. A small fraction of the programme's execution traces is labelled with pass or fail verdicts. Execution traces are then embedded as fixed length vectors and a neural network (NN) component that uses the line-by-line information to classify traces as pass or fail is designed. The classification accuracy of this approach is evaluated using subject programs from different application domains—1. Module from Ethereum Blockchain, 2. Module from PyTorch deep learning framework, 3. Microsoft SEAL encryption library components, 4. Sed stream editor, 5. Nine network protocols from Linux packet identifier, L7-Filter and 6. Utilities library, commons-lang for Java. For all subject programs, it was found that test execution classification had high precision, recall and specificity, averaging to 93%, 94% and 96%, respectively, while only training with an average 14% of the total traces. Experiments show that the proposed NN-based approach is promising in classifying test executions from different application domains.

RESTful APIs are very popular in industry, especially when developing enterprise systems using a microservice architecture. Testing such APIs is challenging, as tests will be composed of not only HTTP calls, but also settings of the environment, like databases. Different blackbox testing techniques have been shown to easily find real faults in many RESTful APIs, with very little human effort from software engineers. However, whitebox techniques could lead to much better results, although having an up-front cost for the engineers. In this paper, we report on the use of the open-source tool EvoMaster, on eight RESTful APIs. We show how EvoMaster can be used to automatically generate test cases that can find several bugs, even when using a naive blackbox approach. When enhancing the search with whitebox information, significantly better results are achieved. However, there are several challenges that need to be taken into account when an engineer wants to use a tool such as EvoMaster to test their projects. This work is funded by the Research Council of Norway (project on Evolutionary Enterprise Testing, grant agreement No 274385).

In the literature, there are clear-cut techniques for system safety analysis but little support for software-specific safety evaluation. This article provides a software safety analysis method to help practitioners comply with ISO 26262-6 in Agile software development.

In recent years, automated software verification has progressed significantly. We can now effectively explore complex software structures through automated testing or to prove properties of complex programs, such as compilers using formal methods. But, for the most part, software testing and formal software verification techniques have advanced independently with relatively few insights on how their research thrusts compare or can be combined.

In the twenty-first century, our life will increasingly depend on software-based products and complex interconnected systems. Thus, the quality and security of software-based systems is a world-wide concern. Combinatorial testing is a versatile methodology for finding errors (bugs) and vulnerabilities in software-based systems. This paper offers a review of combinatorial testing. Combinatorial testing (CT) methods evolved from investigations which looked like factorial experiments (FE) with pass/fail outcomes. We will discuss the similarities and differences between FE and CT. Use of CT for detecting errors (bugs) in software-based systems has gained significant interest from the international software testing community. Many successful results have been reported from the use of CT to detect software errors in aerospace, automotive, defense, cybersecurity, electronic medical systems, and financial service industries. Now, combinatorial testing methods are being increasingly used to investigate vulnerabilities in software-based systems. Combinatorial testing could be useful in detecting errors and security vulnerabilities in Internet of Things, Autonomous Systems, and Artificially Intelligent Software.

Software development is not error-free. For decades, bugs –including physical ones– have become a significant development problem requiring major maintenance efforts. Even in some cases, solving bugs led to increment them. One of the main reasons for bug’s prominence is their ability to hide. Finding them is difficult and costly in terms of time and resources. However, software testing made significant progress identifying them by using different strategies that combine knowledge from every single part of the program. This paper humbly reviews some different approaches from software testing that discover bugs automatically and presents some different state-of-the-art methods and tools currently used in this area. It covers three testing strategies: search-based methods, symbolic execution, and fuzzers. It also provides some income about the application of diversity in these areas, and common and future challenges on automatic test generation that still need to be addressed.

U radu je izvršena analiza automatskog testiranja ADMS funkcionalnosti za učitavanje snimka dinamičkih podataka. Predstavljene su osnovne vrste testiranja softvera, kao i razlike između manuelnog i automatskog testiranja. Opisani su alati i okruženje korišćeni za pisanje automatskih testova. Na kraju rada dati su rezultati izvršavanja napisanih automatskih testova, kao i zaključci do kojih se došlo izvršenom analizom.

Mutation testing is well known as one of the most effective approaches to create test cases, which can detect software faults. However, its drawback is the low scalability — if no special attention is given to improve efficiency — that directly affects its application in practice. This paper shows a replication study focused on emphasizing evidence in which the use of distributed processing structures can improve mutation testing. For this purpose, an architecture that enables mutation testing concurrent execution was designed. Five load balancing algorithms responsible for controlling the distribution and execution of data while carrying out mutation testing were evaluated. Experiments were conducted in order to evaluate the scalability and performance of the architecture considering homogeneous and heterogeneous setups. A time reduction of 50% was observed when executing mutants in parallel in relation to the conventional sequential application of mutation testing. The performance gain was above 95% when there was a higher number of nodes in the distributed architecture.

In today’s Complex era, the need for simplest software application has increased massively. The quality of such a handy application along with adequate testing is the biggest challenge one can face. Software Testing is an integral part of any software development which has to be followed right from the sapling phase of development. This paper focuses on testing methodologies which are used prior along with testing techniques for quality assurance and best of the quality.

Software testing (ST) has been considered as one of the most important and critical activities of the software development life cycle (SDLC) since it influences directly on quality. When a software project is planned, it is common practice to predict the corresponding ST effort (STEP) as a percentage of predicted SDLC effort. However, the effort range for ST has been reported between 10 and 60% of the predicted SDLC effort. This wide range on STEP causes uncertainty in software managers due to STEP is used for allocating resources to teams exclusively for testing activities, and for budgeting and bidding the projects. In spite of this concern, hundreds of studies have been published since 1981 about SDLC effort prediction models, and only thirty-one STEP studies published in the last two decades were identified (just two of them based their conclusions on statistical significance). The contribution of the present study is to investigate the application for STEP of five machine learning (ML) models reported as the most accurate ones when applied to SDLC effort prediction. The models were trained and tested with data sets of projects selected from an international public repository of software projects. The selection for projects was based on their data quality rating, type of development, development platform, programming language generation, sizing method, and resource level of projects. Results based on statistical significance allow suggesting the application of specific ML models to software projects by type of development, and developed on a determined platform and programming language generation.

his article highlights how software simulations can, on the one hand, help uncover bugs, but on the other hand, help exploit them. As a case study, this article details how to exploit bugs in cryptographic hardware.

Deep Neural Networks (DNNs) are being deployed in a wide range of settings today, from safety-critical applications like autonomous driving to commercial applications involving image classifications. However, recent research has shown that DNNs can be brittle to even slight variations of the input data. Therefore, rigorous testing of DNNs has gained widespread attention.While DNN robustness under norm-bound perturbation got significant attention over the past few years, our knowledge is still limited when natural variants of the input images come. These natural variants, e.g., a rotated or a rainy version of the original input, are especially concerning as they can occur naturally in the field without any active adversary and may lead to undesirable consequences. Thus, it is important to identify the inputs whose small variations may lead to erroneous DNN behaviors. The very few studies that looked at DNN’s robustness under natural variants, however, focus on estimating the overall robustness of DNNs across all the test data rather than localizing such error-producing points. This work aims to bridge this gap.To this end, we study the local per-input robustness properties of the DNNs and leverage those properties to build a white-box (DeepRobust-W) and a black-box (DeepRobust-B) tool to automatically identify the non-robust points. Our evaluation of these methods on three DNN models spanning three widely used image classification datasets shows that they are effective in flagging points of poor robustness. In particular, DeepRobust-W and DeepRobust-B are able to achieve an F1 score of up to 91.4% and 99.1%, respectively. We further show that DeepRobust-W can be applied to a regression problem in a domain beyond image classification. Our evaluation on three self-driving car models demonstrates that DeepRobust-W is effective in identifying points of poor robustness with F1 score up to 78.9%.

Mobile and web applications must operate and be displayed correctly on many different devices and browsers. The visual testing of web or mobile applications is usually a manual process that requires a significant amount of testing time, meaning that applications are tested only on a few devices. It is then assumed that the applications will be displayed correctly on other compatible or similar devices. This paper presents an automated visual testing method for user interfaces. The main contributions of this paper are a classification scheme for visual defects of user interfaces and the definition of an automatic visual testing method that tests applications on many different devices with varying hardware and software parameters. The method is based on an automated search for defects using heuristic and expected state prediction algorithms, which involves analyzing the resources used by applications and screenshots. The testing method works by executing applications on a full set of devices, taking a screenshot at every execution step, and analyzing each of these screenshots. The manual as well as automated testing approaches were validated on 781 of Android applications. The experimental results show that the proposed method has advantages over manual testing.

The history of Artificial Intelligence and Machine Learning dates back to 1950’s. In recent years, there has been an increase in popularity for applications that implement AI and ML technology. As with traditional development, software testing is a critical component of an efficient AI/ML application. However, the approach to development methodology used in AI/ML varies significantly from traditional development. Owing to these variations, numerous software testing challenges occur. This paper aims to recognize and to explain some of the biggest challenges that software testers face in dealing with AI/ML applications. For future research, this study has key implications. Each of the challenges outlined in this paper is ideal for further investigation and has great potential to shed light on the way to more productive software testing strategies and methodologies that can be applied to AI/ML applications.

When deadlines and resources of software projects become scarce, testing is usually in the first row to have its activities aborted or reduced. If defects cannot be found, products quality can be affected. In a software development process, aborted or reduced activities that can bring short-term benefits, but can be harmful to the project in a long run, are considered Technical Debt (TD). When TDs impact testing activities, they are called Test Debt. There are several studies dealing with Test Debt, however, current solutions often deal with specific types of tests (e.g., exploratory and automated tests) and do not address the whole software testing process. Aiming to fill these gaps, this paper proposes a Test Debt Catalog with subtypes of Test Debts and technical debt management activities. This catalog was built based on semi-structured interviews conducted with practitioners who perform testing activities in five projects from industry. With our catalog, we intend to help the management of test debts during the execution of software testing processes.

Software Testing may be a method, that involves, death penalty of a software system program/application and finding all errors or bugs therein program/application in order that the result is going to be a defect-free software system. Quality of any software system will solely be acknowledged through means that of testing (software testing). Through the advancement of technology round the world, there inflated the quantity of verification techniques and strategies to check the software system before it goes to production and astray to promote. Automation Testing has created its impact within the testing method. Now-a-days, most of the software system testing is finished with the automation tools that not solely lessens the quantity of individuals operating around that software system however additionally the errors which will be loose through the eyes of the tester. Automation take look acting contains test cases that make the work simple to capture totally different eventualities and store them. Therefore, software system automation testing method plays a significant role within the software system testing success. This study aims in knowing differing kinds of software system testing, software system testing techniques and tools and to match manual testing versus automation testing.

Software testing is a vital part of software engineering process. Automated testing makes this process more accurate and more efficient. For automated testing, many different testing tools were introduced. Due to the large number and the variety of testing tools, selecting the appropriate tools became a difficult confusable task. This research aims at developing a comprehensive taxonomy for testing tools that cover a broad range of testing tools criteria. This comprehensive view would help software developers and software vendors to specify the testing tool/s they need/develop accurately. In details, the framework includes two main parts: (1) comprehensive taxonomy of testing tools; (2) multi-criteria selection method. The first part covers different criteria of testing tools. Because these criteria are large in numbers, wide and variant, a taxonomy of these criteria is needed. This taxonomy will help developers distinguish among testing tools based on a wide spectrum of different criteria. The second part of the framework is a multi-criteria selection method; that enables software developers to choose the appropriate testing tool using a systematic and adequate automated manner. The selection method employs scientific two well-known methods of multi-criteria decision-making techniques; Analytic Hierarchy Process (AHP) and Technique for Order Preference by Similarity to Ideal Solution (TOPSIS). The testing tools taxonomy is well validated by academic professionals in software engineering and achieved good scores in terms of significance, usefulness and comprehension. Academics reported that the taxonomy is slightly complex and needs to be simplified. The selection method was validated using different scenarios to prove the quality of selection even in complex cases with many criteria and many alternatives.

Copyright © 2021 The Author(s). There is an indisputable industrial need for highly skilled individuals in the role of software testers. However, little is known about what activities under testers' responsibilities, competencies, and experiences sought after from employers' perspectives. For the purpose of this research, a data set of 1000 job ads related to software testing role in the United States was collected and analyzed. Specifically, a thorough analysis was conducted to find the industrial demand for competencies for the software testing role in terms of (i) Level of education, training, and experience; (ii) Testing skills; (iii) Technical skills; and (iv) Soft-skills. Relevant correlated skills that may influence shaping the profile of the software tester were also analyzed. Also, the essential duties that a software tester is expected to perform were investigated. The results from the subsequent quantitative and qualitative analysis are reported.

​© 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., Serverless computing aims at reducing processing and operational units to single event-driven functions for service orchestration and choreography. With its micro-granular architectural characteristics, serverless computing is bound to face considerable architectural issues and challenges in the medium- and long-term; are these bound to become Technical Debt? As known to many, technical debt is a metaphor that reflects the additional long-run project costs connected to immediately-expedient but unsavvy technical decisions. However, what does technical debt mean and how is it expressed in serverless computing and other hybrid compute models? This article represents the first attempt to conceptualize Technical Debt in such a context; we base our arguments over a technical overview of serverless computing concepts and practices and elaborate on them via empirical inquiry. Our results suggest that higher serviceability of serverless technologies is also characterized by the absence of mechanisms to support an adequate maintainability, testability, and monitoring of serverless systems. Indeed, in case of unexpected behaviours, testing and maintenance activities are more complex and more expensive, as mainly based on non-automated, manual tasks.

Traceability information can be crucial for software maintenance, testing, automatic program repair, and various other software engineering tasks. Customarily, a vast amount of test code is created for systems to maintain and improve software quality. Today’s test systems may contain tens of thousands of tests. Finding the parts of code tested by each test case is usually a difficult and time-consuming task without the help of the authors of the tests or at least clear naming conventions. Recent test-to-code traceability research has employed various approaches but textual methods as standalone techniques were investigated only marginally. The naming convention approach is a well-regarded method among developers. Besides their often only voluntary use, however, one of its main weaknesses is that it can only identify one-to-one links. With the use of more versatile text-based methods, candidates could be ranked by similarity, thus producing a number of possible connections. Textual methods also have their disadvantages, even machine learning techniques can only provide semantically connected links from the text itself, these can be refined with the incorporation of structural information. In this paper, we investigate the applicability of three text-based methods both as a standalone traceability link recovery technique and regarding their combination possibilities with each other and with naming conventions. The paper presents an extensive evaluation of these techniques using several source code representations and meta-parameter settings on eight real, medium-sized software systems with a combined size of over 1.25 million lines of code. Our results suggest that with suitable settings, text-based approaches can be used for test-to-code traceability purposes, even where naming conventions were not followed.

Testing serverless applications plays an important role in software quality assurance. The current status of testing and debugging in serverless-based applications depicted by the experts helped us highlight issues and challenges that need to be deeply investigated. Post-print / Final draft

Recently, software testing has become a significant component of information security. The most reliable technique for automated software testing is a fuzzing tool that feeds programs with random test-input and detects software vulnerabilities that are critical to security. Similarly, symbolic execution has gained the most attention as an efficient testing tool for producing smart test-inputs and discovering hard-to-reach bugs using search-based heuristics and compositional approaches. The combination of fuzzing and symbolic execution makes software testing more efficient by mitigating the limitations in each other. Although several studies have been conducted on hybrid fuzzing in recent years, a comprehensive and consistent review of hybrid fuzzing techniques has not been explored. To add coherence to the extensive literature on hybrid fuzzing and to make it reach a large audience, this study provides an overview of key concepts along with the taxonomy of existing hybrid fuzzing tools, problems, and solutions that have been developed in this sphere. It also includes evaluations of the proposed approaches and a number of suggestions for the development of hybrid fuzzing in the future.

Fuzzing is a widely used technique to discover vulnerabilities in software. However, for programs requiring highly structured inputs, the byte-based mutation strategies in existing fuzzers have difficulties in generating valid inputs. To resolve this challenge, Grammar-Based Fuzzing (GBF) utilizes existing grammar specifications to generate new inputs. Some GBFs perform mutation based on Abstract Syntax Trees (ASTs), which can generate inputs conforming to grammars. However, the existing GBFs neglect using feedback to optimize mutation strategies, and blindly generate inputs without considering the effectiveness of those inputs. In this paper, we use the power schedule and the subtree pool to optimize mutation strategies. Specifically, we first translate input files into ASTs, and extract subtrees from ASTs into a subtree pool. Then, we optimize the power schedule on AST nodes based on a probabilistic model. That is, we adaptively determine the time budget for mutating an AST node. Finally, we replace AST nodes along with their subtrees using the ones we select from the subtree pool. We implement a fuzzing tool to demonstrate our strategies. The experiment results show that our method outperforms the state-of-the-art methods in fuzzing efficiency.

Recommending effective testers in crowdsourced software testing is a challenge. In this paper, we study the improvement of crowdsourced software testers’ skills over time. We propose the project difficulty coefficient to eliminate the influence of the item on the tester’s score. The hyperbolic learning curve model and exponential learning curve model are used to fit the learning ability of the testers. The experimental results show that when the test data is large, the exponential learning curve can better simulate the improvement of testers’ skills.

The software engineering community is rapidly adopting machine learning for transitioning modern-day software towards highly intelligent and self-learning systems. However, the software engineering community is still discovering new ways how machine learning can offer help for various software development life cycle stages. In this article, we present a study on the use of machine learning across various software development life cycle stages. The overall aim of this article is to investigate the relationship between software development life cycle stages, and machine learning tools, techniques, and types. We attempt a holistic investigation in part to answer the question of whether machine learning favors certain stages and/or certain techniques.