Search

Gamification is an established practice in Software Engineering to increase effectiveness and engagement in many practices. This manuscript provides a characterization of the application of gamification to the Software Testing area. Such practice in fact reportedly suffers from low engagement by both personnel in industrial contexts and learners in educational contexts. Our goal is to identify the application areas and utilized gamified techniques and mechanics, the provided benefits and drawbacks, as well as the open challenges in the field. To this purpose, we conducted a Multivocal Literature Review to identify white and grey literature sources addressing gamified software testing. We analyzed 73 contributions and summarized the most common gamified mechanics, concepts, tools, and domains where they are mostly applied. We conclude that gamification in software testing is mostly applied to the test creation phase with simple white-box unit or mutation testing tools and is mostly used to foster good behaviors by promoting the testers' accomplishment. Key research areas and main challenges in the field are: careful design of tailored gamified mechanics for specific testing techniques; the need for technological improvements to enable crowdsourcing, cooperation, and concurrency; the necessity for empirical and largescale evaluation of the benefits delivered by gamification mechanics. [ABSTRACT FROM AUTHOR]

The software development process has matured significantly over the past decade. We are currently in a state where the need for the verification and validation of the product under development is unquestionable. Visualizations, as software products, must go through such verification and validation control. At the implementation level, a visualization software is no different from any other software, its source code can be verified and validated by using any available white-box technique. The usability studies and controlled experiments are helpful to understand how the user perceives and uses the visualization. However, at an interaction level, a visualization software is not like any other software. Most interactions on a traditional software, are based on buttons and text fields while on a visualization, the interactions are mostly based on zooming, selecting and filtering visual elements. The black-box techniques found in the literature, also known as functional tests, are not suitable for this context. This paper describes a black-box technique tailored for information visualization implementations. The technique is built on constraints imposed over the sequences of low-level interactions available in the visualization and the User Action Notation.

The software development process has matured significantly o ver t he p ast d ecade. W e are currently in a state where the need for the verification and validation of the product under development is unquestionable. Visualizations, as software products, must go through such verification a nd v alidation c ontrol. A t the implementation level, a visualization software is no different from any other software, its source code can be verified a nd v alidated by using any available white-box technique. The usability studies and controlled experiments are helpful to understand how the user perceives and uses the visualization. However, at an interaction level, a visualization software is not like any other software. Most interactions on a traditional software, are based on buttons and text fields w hile o n a v isualization, t he interactions are mostly based on zooming, selecting and filtering visual e lements. The black-box techniques found in the literature, also known as functional tests, are not suitable for this context. This paper describes a black-box technique tailored for information visualization implementations. The technique is built on constraints imposed over the sequences of low-level interactions available in the visualization and the User Action Notation. [ABSTRACT FROM AUTHOR]

Information system security is receiving increasing attention every day because a security problem can cause serious financial loss or even loss of lives. Some of these security problems occur as a result of poor design practices, where important security functionality is not designed properly and is directly implemented later in the development cycle in an unmethodical way. Researchers have put a great deal of effort into defining processes and tools to design and develop more secure information systems. However, verification of the designed and developed security functionality is of utmost importance. In some cases, designs and codes also need to be formally or semi-formally verified and certified by authorities. The Common Criteria is one of the widely used universal frameworks for evaluating the security functionality of information systems. In this paper, we propose a new framework, model driven security framework, for the analysis, design, and evaluation of security properties of information systems. Our aim is to support information system developers and evaluation authorities who implement the higher-level Common Criteria (levels 6 and 7) security assurance process using formal methods based on Unified Modelling Language, Object Constraint Language, Promela, and Spin. Copyright © 2015 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

Certifying safety-critical software and ensuring its safety requires checking the conformance between safety requirements and design. Increasingly, the development of safety-critical software relies on modeling, and the System Modeling Language (SysML) is now commonly used in many industry sectors. Inspecting safety conformance by comparing design models against safety requirements requires safety inspectors to browse through large models and is consequently time consuming and error-prone. To address this, we have devised a mechanism to establish traceability between (functional) safety requirements and SysML design models to extract design slices (model fragments) that filter out irrelevant details but keep enough context information for the slices to be easy to inspect and understand. In this article, we report on a controlled experiment assessing the impact of the traceability and slicing mechanism on inspectors' conformance decisions and effort. Results show a significant decrease in effort and an increase in decisions' correctness and level of certainty. [ABSTRACT FROM AUTHOR]

In this article, the authors give an overview of tool-based verification of hardware and software systems and discuss the relation between verification and logical reasoning. Here, "verification"' refers to reasoning-based methods to establish dependability. This isn't restricted to proofs of functional correctness; it also includes other scenarios such as test generation and bug finding. The authors describe the main verification scenarios and methods that are in use today and the extent to which they depend on logical reasoning. From this discussion, they distill current trends and new opportunities for the interaction between verification and reasoning. [ABSTRACT FROM AUTHOR]

For this special issue, the guest editors asked a panel of six established experts in software analytics to highlight what they thought were the most important, or overlooked, aspect of this field. They all pleaded for a much broader view of analytics than seen in current practice: software analytics should go beyond developers (Ahmed Hassan) and numbers (Per Runeson). Analytics should also prove its relevance to practitioners (Abram Hindle, Martin Shepperd). There are now opportunities for "natural" software analytics based on statistical natural language processing (Prem Devanbu). Lastly, software analytics needs information analysts and field agents like Chloe O'Brian and Jack Bauer in the TV show 24 (Sung Kim). [ABSTRACT FROM PUBLISHER]

Published scientific research that relies on numerical computations is too often not reproducible. For computational research to become consistently and reliably reproducible, the process must become easier to achieve, as part of day-to-day research. A combination of best practices and automated tools can make it easier to create reproducible research. [ABSTRACT FROM AUTHOR]

Recent advances in mechanical techniques for systematic testing have increased our ability to automatically find subtle bugs, and hence, to deploy more dependable software. This paper builds on one such systematic technique, scope-bounded testing, to develop a novel specification-based approach for efficiently generating tests for products in a software product line. Given properties of features as first-order logic formulas in Alloy, our approach uses SAT-based analysis to automatically generate test inputs for each product in a product line. To ensure soundness of generation, we introduce an automatic technique for mapping a formula that specifies a feature into a transformation that defines incremental refinement of test suites. Our experimental results using different data structure product lines show that an incremental approach can provide an order of magnitude speedup over conventional techniques. We also present a further optimization using dedicated integer constraint solvers for feature properties that introduce integer constraints, and show how to use a combination of solvers in tandem for solving Alloy formulas. [ABSTRACT FROM AUTHOR]

Most works that extend workflow validation beyond syntactical checking consider constraints on the sequence of messages exchanged between services. These constraints are expressed only in terms of message names and abstract away their actual data content. We provide examples of real-world "data-aware" Web service constraints where the sequence of messages and their content are interdependent. To this end, we present CTL-FO+ an extension over Computation Tree Logic that includes first-order quantification on message content in addition to temporal operators. We show how CTL-FO+ is adequate for expressing data-aware constraints, give a sound and complete model checking algorithm for CTL-FO+ and establish its complexity to be PSPACE-complete. A "naive" translation of CTL-FO+ into CTL leads to a serious exponential blowup of the problem that prevents existing validation tools to be used. We provide an alternate translation of CTL-FO+ into CTL, where the construction of the workflow model depends on the property to validate. We show experimentally how this translation is significantly more efficient for complex formulas and makes model checking of data-aware temporal properties on real-world Web service workflows tractable using off-the-shelf tools. [ABSTRACT FROM AUTHOR]

In many IP media services, the media channels are point-to-point, dynamic and set up with the participation of one or more application servers, even though the media packets themselves travel directly between media end points. The application sewers must be programmed so that the media behavior is globally correct, even though the servers may attempt to manipulate the same media channels concurrently and without knowledge of each other. Our proposed solution to this problem of compositional media control includes an architecture-independent descriptive model, a set of high-level programming primitives, a formal specification of their compositional semantics, a signaling protocol, an implementation, and a partial verification of correctness. The paper includes a performance analysis, a comparison to related work, and principles for making other networked applications more compositional. [ABSTRACT FROM AUTHOR]

We present Delta Execution, a technique that speeds up state-space exploration of object-oriented programs. State-space exploration is the essence of model checking and an increasingly popular approach for automating test generation. A key issue in exploration of object-oriented programs is handling the program state, in particular, the heap. We exploit the fact that many execution paths in state-space exploration partially overlap. Delta Execution simultaneously operates on several states/heaps and shares the common parts across the executions, separately executing only the "deltas" where the executions differ. We implemented Delta Execution in two model checkers: JPF, a popular general-purpose model checker for Java programs, and BOX, a specialized model checker that we developed for efficient exploration of sequential Java programs. The results of bounded-exhaustive exploration of 10 basic subject programs and one larger case study show that Delta Execution reduces exploration time from 1.06x to 126.80x (with median 5.60x) in JPF and from 0.58x to 4.16x (with median 2.23x) in BOX. The results of nonexhaustive exploration in JPF show that Delta Execution reduces exploration time from 0.92x to 6.28x (with median 4.52x). [ABSTRACT FROM AUTHOR]

This work is part of a series referring to COMDEVALCO - a framework for Software Component Definition, Validation, and Composition. Its constituents are: a modeling language, a component repository and a set of tools. This is the first paper describing the object-oriented modeling language, which contains finegrained constructions, aimed to give a precise description of software components. The current status of the language reflects the procedural paradigm constructs. [ABSTRACT FROM AUTHOR]

Web applications are widely adopted and their correct functioning is mission critical for many businesses, At the same time, Web applications tend to be error prone and implementation vulnerabilities are readily and commonly exploited by attackers. The design of countermeasures that detect or prevent such vulnerabilities or protect against their exploitation is an important research challenge for the fields of software engineering and security engineering. In this paper, we focus on one specific type of implementation vulnerability, namely, broken dependencies on session data. This vulnerability can lead to a variety of erroneous behavior at runtime and can easily be triggered by a malicious user by applying attack techniques such as forceful browsing, This paper shows how to guarantee the absence of runtime errors due to broken dependencies on session data in Web applications. The proposed solution combines development-time program annotation, static verification, and runtime checking to provably protect against broken data dependencies. We have developed a prototype implementation of our approach, building on the JML annotation language and the existing static verification tool ESC/Java2, and we successfully applied our approach to a representative J2EE-based e-commerce application. We show that the annotation overhead is very small, that the performance of the fully automatic static verification is acceptable, and that the performance overhead of the runtime checking is limited. [ABSTRACT FROM AUTHOR]

We describe an extension of the SPIN model checker for use on multicore shared-memory systems and report on its performance. We show how, with proper load balancing, the time requirements of a verification run can, in some cases, be reduced close to N-fold when N processing cores ere used. We also analyze the types of verification problems for which multicore algorithms cannot provide relief. The extensions discussed here require only relatively small changes in the SPIN source code and are compatible with most existing verification modes such as partial order reduction, the verification of temporal logic formulas, bitstate hashing, and hash-compact compression. [ABSTRACT FROM AUTHOR]

In this paper, we describe a methodology integrating verification and conformance testing. A specification of a system--an extended input-output automaton, which may be infinite-state---and a set of safety properties ("nothing bad ever happens") and possibility properties ("something good may happen") are assumed. The properties are first tentatively verified on the specification using automatic techniques based on approximated state-space exploration, which are sound, but, as a price to pay for automation, are not complete for the given class of properties. Because of this incompleteness and of state-space explosion, the verification may not succeed in proving or disproving the properties. However, even if verification did not succeed, the testing phase can proceed and provide useful information about the implementation. Test cases are automatically and symbolically generated from the specification and the properties and are executed on a black-box implementation of the system. The test execution may detect violations of conformance between implementation and specification; in addition, it may detect violation/satisfaction of the properties by the implementation and by the specification. In this sense, testing completes verification. The approach is illustrated on simple examples and on a Bounded Retransmission Protocol. [ABSTRACT FROM AUTHOR]

Background: Open source (OS) software is continuously gaining recognition and use in the biomedical domain, for example, in health informatics and bioinformatics. Objectives: Given the mission critical nature of applications in this domain and their potential impact on patient safety, it is important to understand to what degree and how effectively biomedical OS developers perform standard quality assurance (QA) activities such as peer reviews and testing. This would allow the users of biomedical OS software to better understand the quality risks, if any, and the developers to identify process improvement opportunities to produce higher quality software. Methods: A survey of developers working on biomedical OS projects was conducted to examine the QA activities that are performed. We took a descriptive approach to summarize the implementation of QA activities and then examined some of the factors that may be related to the implementation of such practices. Results: Our descriptive results show that 63% (95% CI, 54-72) of projects did not include peer reviews in their development process, while 82% (95% CI, 75-89) did include testing. Approximately 74% (95% CI, 67-81) of developers did not have a background in computing, 80% (95% CI, 74-87) were paid for their contributions to the project, and 52% (95% CI, 43-60) had PhDs. A multivariate logistic regression model to predict the implementation of peer reviews was not significant (likelihood ratio test = 16.86, 9 df, P =.051) and neither was a model to predict the implementation of testing (likelihood ratio test = 3.34, 9 df, P =.95). Conclusions: Less attention is paid to peer review than testing. However, the former is a complementary, and necessary, QA practice rather than an alternative. Therefore, one can argue that there are quality risks, at least at this point in time, in transitioning biomedical OS software into any critical settings that may have operational, financial, or safety implications. Developers of biomedical OS applications should invest more effort in implementing systemic peer review practices throughout the development and maintenance processes. [ABSTRACT FROM AUTHOR]

The problem of generating a checking experiment from a nondeterministic finite state machine has been represented in terms of state counting. However, test techniques that use state counting traditionally produce preset test suites. This paper extends the notion of state counting in order to allow the input/output sequences observed in testing to be utilized: Adaptive state counting is introduced. The main benefit of the proposed approach is that it may result in a reduction in the size of the test suite used. An additional benefit is that, where a failure is observed, it is possible to terminate test generation at this point. [ABSTRACT FROM AUTHOR]

The development of robust software is a difficult undertaking and is becoming increasingly more important as applications grow larger and more complex. Although modem programming languages such as C++ and Java provide sophisticated exception handling mechanisms to detect and correct runtime error conditions, exception handling code must still be programmed with care to preserve application consistency. In particular, exception handling is only effective if the premature termination of a method due to an exception does not leave an object in an inconsistent state. We address this issue by introducing the notion of failure atomicity in the context of exceptions. We propose practical techniques to automatically detect and mask the nonatomic exception handling situations encountered during program execution. These techniques can be applied to applications written in various programming languages that support exceptions. We perform experimental evaluation on both C++ and Java applications to demonstrate the effectiveness of our techniques and measure the overhead that they introduce. [ABSTRACT FROM AUTHOR]

Advanced computer architectures rely mainly on compiler optimizations for parallelization, vectorization, and pipelining. Efficient code generation is based on a control dependence analysis to find the basic blocks and to determine the regions of control. However, unstructured branch statements, such as jumps and goto's, render the control flow analysis difficult, time-consuming, and result in poor code generation. Branches are part of many programming languages and occur in legacy and maintenance code as well as in assembler, intermediate languages, and byte code. A simple and effective technique is presented to convert unstructured branches into hammock graph control structures. Using three basic transformations, an equivalent program is obtained in which all control statements have a well-defined scope. In the interest of predication and branch prediction, the number of control variables has been minimized, thereby allowing a limited code replication. The correctness of the transformations has been proven using an axiomatic proof rule system. With respect to previous work, the algorithm is simpler and the branch conditions are less complex, making the program more readable and the code generation more efficient. Additionally, hammock graphs define single entry single exit regions and therefore allow localized optimizations. The restructuring method has been implemented into the parallelizing compiler FPT and allows to extract parallelism in unstructured programs. The use of hammock graph transformations in other application areas such as vectorization, decompilation, and assembly program restructuring is also demonstrated. [ABSTRACT FROM AUTHOR]

In this paper we present a case-study in which the tool Uppaal is extended and applied to verify an audio-control protocol developed by Philips. The size of the protocol studied in this paper is significantly larger than case studies, including various abstract versions of the same protocol without bus-collision handling, reported previously in the community of real-time verification. We have checked that the protocol will function correctly if the timing error of its components is bound to ±5%, and incorrectly if the error is ±6%. In addition, using Uppaal’s ability of generating diagnostic traces, we have studied an erroneous version of the protocol actually implemented by Philips, and constructed a possible execution sequence explaining the error. During the case-study, Uppaal was extended with the notion of committed locations. It allows for accurate modelling of atomic behaviours, and more importantly, it is utilised to guide the state-space exploration of the model checker to avoid exploring unnecessary interleavings of independent transitions. Our experimental results demonstrate considerable time and space-savings of the modified model checking algorithm. In fact, due to the huge time and memory-requirement, it was impossible to check a simple reachability property of the protocol before the introduction of committed locations, and now it takes only seconds. [ABSTRACT FROM AUTHOR]