Search

Safety and security are important properties of any software system, particularly in safety-critical domains, such as embedded, automotive, or cyber-physical systems. Moreover, particularly those domains also employ highly-configurable systems to customize variants, for example, to different customer requirements or regulations. Unfortunately, we are missing an overview understanding of what research has been conducted on the intersection of safety and security with configurable systems. To address this gap, we conducted a systematic mapping study based on an automated search, covering ten years (2011--2020) and 65 relevant (out of 367) publications. We classified each publication based on established security and safety concerns (e.g., CIA triad) as well as the connection to configurable systems (e.g., ensuring security of such a system). In the end, we found that considerably more research has been conducted on safety concerns, but both properties seem under-explored in the context of configurable systems. Moreover, existing research focuses on two directions: Ensuring safety and security properties in product-line engineering; and applying product-line techniques to ensure safety and security properties. Our mapping study provides an overview of the current state-of-the-art as well as open issues, helping practitioners identify existing solutions and researchers define directions for future research.

An autonomous car is a driverless vehicle for the purpose of navigating autonomously and safely over the earth's surface. Interest in this type of transport has increased significantly in recent years and it is possible that the number of these vehicles on the streets will increase, changing the daily lives of cities. In this context, this work aims to understand the technological limitations and impacts of using autonomous cars, observing the feasibility, adaptation, traffic legislation, and perceived benefits. Following the method of systematic literature review proposed by Kitchenham, 69 primary studies from relevant bases in the area of Technology were analyzed. These studies showed some evidence by indicating, for example, that there will be a change in the way people relate to cars, particularly in terms of sharing them, in addition to impacts on urban mobility.

Smart meters and the smart grid will allow utility companies and customers to monitor their electricity and utility usage in fine-grained detail instead of the previously common monthly or yearly measurements. With this fine-grained detail comes serious privacy concerns. One of the most promising solutions for measuring and preserving privacy loss is differential privacy. Both differential privacy and the smart grid are relatively young developments that will require more research before they can be confidently implemented worldwide. With this systematic mapping study, we will provide an overview of the current literature and attempt to determine the future directions the research may take.

Deep Learning (DL) is being used nowadays in many traditional Software Engineering (SE) problems and tasks. However, since the renaissance of DL techniques is still very recent, we lack works that summarize and condense the most recent and relevant research conducted at the intersection of DL and SE. Therefore, in this paper, we describe the first results of a mapping study covering 81 papers about DL & SE. Our results confirm that DL is gaining momentum among SE researchers over the years and that the top-3 research problems tackled by the analyzed papers are documentation, defect prediction, and testing., 8 pages, 5 figures, 4 tables. Accepted for publication at ACM SAC 2021

As opposed to traditional software development, agile development encourages requirements change. Given the importance of requirements change management, there is an increasing number of studies discussing this process in both traditional and agile software developments. By examining existing works, we observed that there are many ways to improve the management of agile requirements change. Aiming to addressing this research gap, our study focus on defining a process to handle Agile Requirement Change Management (ARCM), as well as identifying practices to support ARCM process. We performed systematic study to map the main practices applied to manage agile requirements change. Our primary goal was to seek a better understanding of possible directions in the ARCM research, and foster discussions for future work. We identified 3 steps to handle the ARCM process, and grouped 11 distinct agile practices to carry out these steps. Even though agile practices might be, to some extent, for managing requirement change, we were able to identify practical challenges when adopting these practices for Agile Requirements Changes management.

[Context] Software projects must consider the selection of testing techniques and criteria during their life cycles. This practice increases the chances of testing activity to be appropriately performed. In a previous work, an infrastructure to support the selection of testing techniques was proposed for the context of concurrent software. This infrastructure considers information (attributes) of the project to make the selection closer to the testers need. [Objective] This paper extends the previous work by identifying new studies concerning testing techniques selection, project attributes that can be used for this selection and which approaches can be employed to support the combined selection. [Method] A mapping study was conducted and a total of 15 primary studies, published in the last 20 years were selected. Information about approaches to testing techniques selection was analyzed and classified. [Results] The following results were obtained: (i) existing approaches for selection of testing techniques; (ii) proposition of a taxonomy of selection approaches; (iii) characterization of attributes to offer the support for the selection of a testing technique; and (iv) identification of approaches that perform combined selection of testing techniques. [Conclusion] Combining testing techniques is essential to improve the testing activity quality by finding different failure categories and supplementing other techniques limitations. This paper describes an initiative to offer support for the construction of new or combined strategies for the selection of testing techniques capable of being used in practice.

Reuse has been recognized as an important practice in software engineering. The use of patterns makes it easier to reuse successful solutions, speeds up the development process, and promotes the application of good practices. Related patterns can be organized in a Pattern Language (PL), which represents the patterns and their relations, and provides guidance on how to select, reuse and integrate them. Visual notations are often used to provide a graphical representation to PLs. Aiming to investigate how PLs related to software have been visually represented, we carried out a systematic mapping. We identified and analyzed 64 PLs. As a result, we noticed a lack of consensus on the elements that should be represented in a PL and the symbols used to represent them. Moreover, most PLs have ambiguous or inexpressive visual representations.

The teaching-learning process of Software Engineering (SE) has been widely discussed by several Brazilian authors. This concern reflects the demand for qualified professionals, since quality in teaching can contribute to the improvement of software development. To do so, there is a need to move beyond the expository format to incorporate new approaches and techniques. From a systematic mapping of the literature in Brazil, which included an automatic search in Google Scholar, manual searches in two journals and four important Brazilian events in the area of computing education, complemented by snowballing, articles with proposals for games, tools, teaching methodologies and integration between disciplines were identified and analyzed to support the software engineering education. The results contribute to the evaluation of the state of the art in the area.

Higher Order Mutation Testing (HOMT) uses mutants derived by applying mutation operators more than once in the program under test. This kind of test increases efficacy, but can be more expensive and lead to some challenges such as the large space of mutants. The challenges can be managed in the Search-Based Software Engineering field to seek only interesting mutants that satisfy some desired conditions regarding, for example, type of revealed faults or reduction of equivalent mutants. For this reason, we observe a growing interest in search-based HOMT and an increasing number of related studies. To better characterize such studies, this paper presents results of a mapping on search-based HOMT. We found 25 primary studies on this subject and identified the most common search-based algorithms, preferred fitness functions, addressed programming languages, and evaluation aspects. In addition to this, some trends and research opportunities are identified that allow researchers to direct future investigations.

The Blackbox project has been collecting programming activity data from users of BlueJ (a novice-targeted Java development environment) for nearly five years. The resulting dataset of more than two terabytes of data has been made available to interested researchers from the outset. In this paper, we assess the impact of the Blackbox project: we perform a mapping study to assess eighteen publications which have made use of the Blackbox data, and we report on the advantages and difficulties experienced by researchers working with this data, collected via a survey. We find that Blackbox has enabled pieces of research which otherwise would not have been possible, but there remain technical challenges in the analysis. Some of these -- but not all -- relate to the scale of the data. We provide suggestions for the future use of Blackbox, and reflections on the role of such data collection projects in programming research.

Background. Continuous experimentation (CE) has recently emerged as an established industry practice and as a research subject. Our aim is to study the application of CE and A/B testing in various industrial contexts. Objective. We wanted to investigate whether CE is used in different sectors of industry, by how it is reported in academic studies. We also wanted to explore the main topics researched to give an overview of the subject and discuss future research directions. Method. We performed a systematic mapping study of the published literature and included 62 papers, using a combination of database search and snowballing. Results. Most reported software experiments are done online and with software delivered as a service, although varied exemptions exist for e.g., financial software and games. The most frequently researched topics are challenges to conduct experiments and statistical methods for software experiments. Conclusions. The software engineering research on CE is still in its infancy. There are future research opportunities in evaluation research of technical topics and investigations of ethical experimentation. We conclude that the included studies show that A/B testing is applicable to a diversity of software and organisations.

The ability to predict student performance in a course or program creates opportunities to improve educational outcomes. With effective performance prediction approaches, instructors can allocate resources and instruction more accurately. Research in this area seeks to identify features that can be used to make predictions, to identify algorithms that can improve predictions, and to quantify aspects of student performance. Moreover, research in predicting student performance seeks to determine interrelated features and to identify the underlying reasons why certain features work better than others. This working group report presents a systematic literature review of work in the area of predicting student performance. Our analysis shows a clearly increasing amount of research in this area, as well as an increasing variety of techniques used. At the same time, the review uncovered a number of issues with research quality that drives a need for the community to provide more detailed reporting of methods and results and to increase efforts to validate and replicate work.

Search phase is considered as one of the most important steps in conducting secondary studies such as systematic literature reviews and mapping studies. In recent times, automatic search in digital libraries and academic search engines has been the preferred method of search phase execution for most software engineering related secondary studies. However, there are no previous studies that report or evaluate the secondary study relevant search features of these electronic data sources. We perform a feature analysis (screening mode) based evaluation of five widely used digital libraries (IEEE Xplore, ACM DL, SpringerLink, ScienceDirect and Wiley) in terms of their respective features required to support the search phase of secondary studies. We identify a total of 68 search related features and conduct a comprehensive exploration into their execution behaviors. The overall work presents a useful caution guide for systematic reviewers who plan to use the identified features for executing various search phase steps of their secondary studies.

Context: We revisit our review of data quality within the context of empirical software engineering eight years on from our PROMISE 2008 article. Objective: To assess the extent and types of techniques used to manage quality within data sets. We consider this a particularly interesting question in the context of initiatives to promote sharing and secondary analysis of data sets. Method: We update the 2008 mapping study through four subsequently published reviews and a snowballing exercise. Results: The original study located only 23 articles explicitly considering data quality. This picture has changed substantially as our updated review now finds 283 articles, however, our estimate is that this still represents perhaps 1% of the total empirical software engineering literature. Conclusions: It appears the community is now taking the issue of data quality more seriously and there is more work exploring techniques to automatically detect (and sometimes repair) noise problems. However, there is still little systematic work to evaluate the various data sets that are widely used for secondary analysis; addressing this would be of considerable benefit. It should also be a priority to work collaboratively with practitioners to add new, higher quality data to the existing corpora.

Context. Software Product Lines (SPL) has become one of the most prominents way to promote the systematic reuse of software artifacts. Like any other piece of software, with the SPL aging, it becomes necessary to manage their evolution. However, in this process, engineers might introduce divergences among the SPL artifacts. Thus, a number of initiatives address the management of such inconsistencies. Objective. In this paper, we mapped the existing approaches to inconsistency management within SPL. Method. We used the systematic mapping study methodology. Results. We classified and performed a characterization of the approaches found, which we mangaged to arrange in three main categories. Most papers selected proposed new methods as solution research. Besides, there is still a need for validation and evaluation studies. Conclusion. We identified a lack of support for a number of activities of consistency assurance. For instance, no paper addressed the tracking of findings, decisions, and actions, as well as, few papers describing either the handling or a management policy for identified inconsistencies.

Context: Juristo et al. [7] published a literature review about testing technique experiments. The goal was to provide a picture of which techniques and aspects of techniques had been studied experimentally, and try to compile a body of knowledge on testing techniques. Goal: In this paper, we extend Juristo et al.'s study to cover the years from 2000 (where it ended) until 2013. Method: We have performed a systematic mapping study. Results: The situation in testing experimentation has not changed since Juristo et al.'s study. Conclusions: The research field has the same shortcomings.

Bayesian Networks (BN) have been used for decision making in software engineering for many years. We investigate the current status of BNs in predicting software quality in three aspects: 1) techniques used for parameter learning, 2) techniques used for structure learning, and 3) type of variables that represent BN nodes. We performed a systematic mapping study on 38 primary studies that employed BNs to predict software quality. The most popular technique for building the final structure of BNs is the use of expert knowledge with different inference algorithms. Variables in BNs are treated as categorical in more than 70% of studies. Compared to other domains, the usage of BNs is still very limited due to high dependency on expert knowledge and tools.

Context. A recent mapping study intended to verify the current state of replication of empirical studies in Software Engineering (SE) identified two sets of studies: the empirical studies reporting actually replications and a second group of studies that were concerned with concepts, classifications, guidelines, and other themes about replication. Objective. The purpose of this research is to analyze and discuss the content of the second set of studies about replications published between 1996 and 2013. Method. This is a mapping study, in which the primary studies were collected by two previous mapping studies covering the period 1996-2012 complemented by a non-systematic search that collected some articles published in 2013. Results. We analyzed 36 papers reporting empirical and non-empirical studies about replications in SE published in the last 17 years. These papers explored different topics related to concepts and classifications, presented guidelines, and discussed theoretical issues that are relevant for our understanding of replication in our field.

Context: Software processes have become inherently complex to cope with the various situations we face in industrial project environments. In response to this problem, the research area of Method Engineering arose in the 1990s aiming at the systematization of process construction. Objective: Although the research area has gained much attention and offered a plethora of contributions so far, we still have little knowledge about the feasibility of Method Engineering. To overcome this shortcoming, necessary is a systematic investigation of the respective publication flora. Method: We conduct a systematic mapping study and investigate, inter alia, which contributions were made over time and which research type facet they address to distill a common understanding of the state-of-the-art. Results: Based on the review of 64 publications, our results show that most of those contributions only repeat and discuss formerly introduced concepts, whereas empirically sound evidence on the feasibility of Method Engineering, is still missing. Conclusion: Although the research area constitutes many contributions, yet missing are empirically sound investigations that would allow for practical application and experience extraction.

Context: At the same time as our dependence on IT systems increases, the number of reports of problems caused by failures of critical IT systems has also increased. This means that there is a need for risk analysis in the development of this kind of systems. Risk analysis of technical systems has a long history in mechanical and electrical engineering. Objective: Even if a number of methods for risk analysis of technical systems exist, the failure behaviour of information systems is typically very dierent from mechanical systems. Therefore, risk analysis of IT systems requires dierent risk analysis techniques, or at least adaptations of traditional approaches. This means that there is a need to understand what types of methods are available for IT systems and what research that has been conducted on these methods. Method: In this paper we present a systematic mapping study on risk analysis for information systems. 1086 unique papers were identied in a database search and 57 papers were identied as relevant for this study. These papers were classied based on 5 dierent criteria. Results: This classication, for example, shows that most of the discussed risk analysis methods are qualitative and not quantitative and that most of the risk analysis methods that are presented in these papers are developed for IT systems in general and not for specic types of IT system, like medical systems. Conclusions: The results show that many new risk analysis methods have been proposed in the last decade but even more that there is a need for more empirical evaluations of the dierent risk analysis methods. Many papers were identified that propose new risk analysis methods, but few papers discuss a systematic evaluation of these methods or a comparison of dierent methods based on empirical data.

Context: Although the Unified Modeling Language (UML) is now widely used, there is little empirical knowledge about the effectiveness of its different elements.Objectives: To conduct a systematic review of the literature describing empirical studies of this paradigm.Method: We undertook a Mapping Study of the literature, based upon electronic searching of major digital libraries.Results: 33 papers have been identified and classified by topic, form of study involved, and type of publication.Conclusions: The largest group of empirical studies of the UML concentrate on comprehension, and many key aspects have hardly been investigated at all.

IoT systems are typically distributed and performing coordinated behavior across IoT, edge and cloud infrastructures. To fully realize the great potential of IoT systems, it is important to facilitate their creation and operation. It is crucial to have a clear picture of the research landscape of the existing approaches and tools for supporting the deployment and/or orchestration of IoT systems (Depo4IoT). Such a picture can show us how advanced the current state of the art is and what are the gaps to address. We conducted a systematic mapping study (SMS) to find out the research landscape in this domain. The results of our SMS show a sharp increase in the number of primary Depo4IoT publications in two recent years. We found that most approaches do not really support the deployment or orchestration at IoT devices level. Finally, we suggest some potential research directions to address the research gaps that have been found.