Search

Context: Software testing is the most used technique for quality assurance in industry. However, in computer science education software testing is still treated as a second-class citizen and students are unable to test their software well enough. One reason for this is that teaching the subject of software testing is difficult as it is a complex intellectual activity for which students need to allocate multiple cognitive resources at the same time. A myriad of primary and secondary studies have tried to solve this problem in education, however still with very limited results. Objective: Before we can design interventions to improve our pedagogical approaches, we need to gain more in-depth understanding and recognition of sensemaking as it is happening when students design test cases. Method: An initial exploratory study identified four different sensemaking approaches used by students while creating test models. In this paper we present a follow-up study with 50 students from a large university in Spain. The used methodology was based on the previous study with the improvements that originated from its evaluation. We asked the participants to create a test model based on a description of a test problem using a specialized web-based tool for modeling test cases. We measured how well these models fit the test problem, the sensemaking process that students went through when creating the models, and the students’ perception of the modeling task. The participants received no compensation for their efforts, and we scheduled the experiment during a regular class. Apart from the created models and their metadata, we also collected recordings of the students’ computer screens made during the experiment and used a questionnaire to study their perspectives on the assignment. All the collected textual, graphical, and video data was analyzed using an iterative inductive analysis process to allow new information about the different sensemaking approaches to emerge. Results: We gained better insights into the sensemaking processes of students while modeling test cases for a problem. The results enabled us to refine our previous findings, and we identified new sensemaking approaches. Conclusions: Based on these results, we can further investigate ways to influence the sensemaking process in education, the possible misconceptions that have a negative influence on it, and the desired mental model we want our students to have to design test cases.

Background Diabetes is one of the leading causes of death and disability worldwide, and affects people regardless of country, age group, or sex. Using the most recent evidentiary and analytical framework from the Global Burden of Diseases, Injuries, and Risk Factors Study (GBD), we produced location-specific, age-specific, and sex-specific estimates of diabetes prevalence and burden from 1990 to 2021, the proportion of type 1 and type 2 diabetes in 2021, the proportion of the type 2 diabetes burden attributable to selected risk factors, and projections of diabetes prevalence through 2050. Methods Estimates of diabetes prevalence and burden were computed in 204 countries and territories, across 25 age groups, for males and females separately and combined; these estimates comprised lost years of healthy life, measured in disability-adjusted life-years (DALYs; defined as the sum of years of life lost [YLLs] and years lived with disability [YLDs]). We used the Cause of Death Ensemble model (CODEm) approach to estimate deaths due to diabetes, incorporating 25 666 location-years of data from vital registration and verbal autopsy reports in separate total (including both type 1 and type 2 diabetes) and type-specific models. Other forms of diabetes, including gestational and monogenic diabetes, were not explicitly modelled. Total and type 1 diabetes prevalence was estimated by use of a Bayesian meta-regression modelling tool, DisMod-MR 2.1, to analyse 1527 location-years of data from the scientific literature, survey microdata, and insurance claims; type 2 diabetes estimates were computed by subtracting type 1 diabetes from total estimates. Mortality and prevalence estimates, along with standard life expectancy and disability weights, were used to calculate YLLs, YLDs, and DALYs. When appropriate, we extrapolated estimates to a hypothetical population with a standardised age structure to allow comparison in populations with different age structures. We used the comparative risk assessment framework to estimate the risk-attributable type 2 diabetes burden for 16 risk factors falling under risk categories including environmental and occupational factors, tobacco use, high alcohol use, high body-mass index (BMI), dietary factors, and low physical activity. Using a regression framework, we forecast type 1 and type 2 diabetes prevalence through 2050 with Socio-demographic Index (SDI) and high BMI as predictors, respectively. Findings In 2021, there were 529 million (95% uncertainty interval [UI] 500–564) people living with diabetes worldwide, and the global age-standardised total diabetes prevalence was 6·1% (5·8–6·5). At the super-region level, the highest age-standardised rates were observed in north Africa and the Middle East (9·3% [8·7–9·9]) and, at the regional level, in Oceania (12·3% [11·5–13·0]). Nationally, Qatar had the world’s highest age-specific prevalence of diabetes, at 76·1% (73·1–79·5) in individuals aged 75–79 years. Total diabetes prevalence—especially among older adults—primarily reflects type 2 diabetes, which in 2021 accounted for 96·0% (95·1–96·8) of diabetes cases and 95·4% (94·9–95·9) of diabetes DALYs worldwide. In 2021, 52·2% (25·5–71·8) of global type 2 diabetes DALYs were attributable to high BMI. The contribution of high BMI to type 2 diabetes DALYs rose by 24·3% (18·5–30·4) worldwide between 1990 and 2021. By 2050, more than 1·31 billion (1·22–1·39) people are projected to have diabetes, with expected age-standardised total diabetes prevalence rates greater than 10% in two super-regions: 16·8% (16·1–17·6) in north Africa and the Middle East and 11·3% (10·8–11·9) in Latin America and Caribbean. By 2050, 89 (43·6%) of 204 countries and territories will have an age-standardised rate greater than 10%.

People and society are getting more and more dependent on software since it increasingly determines our daily life activities, either in social or business contexts. With the increasing use of software, comes an increasing need for programmers that develop that software. Learning how to program is a complex process that goes beyond the learning of the syntax of a programming language. Students need to learn how to solve problems with computational thinking. Practicing logical reasoning and problem-solving are at the centre of the activities performed by programmers. Thus, the best way to learn computational thinking is by practicing a lot, and one of the most fun ways to do that is by using games.Gamification has demonstrated to be an effective technique to motivate and enact students with learning. For that reason, gamification has been used in all kinds of areas, including teaching of programming. In this work we present a simple browser game, called GIPPPY, to help students that are starting to learn programming in the practice of computational thinking skills. To do that, the game has 12 levels that go from easy to more complex programming concepts. Taking into account the massive use of Python as a programming language in the last years, we have selected the Python language to design the challenges of the game. Therefore, GIPPPY presents the programming concepts using Python notation.To check the impact of this game, we have evaluated GIPPPY with engineering students of the Universitat Politécnica de Valencia during the second semester of 2021. Results show that students had fun when learning, and that students consider the game as a good complement to the currently used techniques to start practicing computational thinking and learning Python.

Computer games are very challenging to handle for traditional automated testing algorithms. In this paper we will look at intelligent agents as a solution. Agents are suitable for testing games, since they are reactive and able to reason about their environment to decide the action they want to take. This paper presents the experience of using an agent-based automated testing framework called iv4xrto test computer games. Three games will be discussed, including a sophisticated 3D game called Space Engineers. We will show how the framework can be used in different ways, either directly to drive a test agent, or as an intelligent functionality that can be driven by a traditional automated testing algorithm such as a random algorithm or a model based testing algorithm. &nbsp

Modern interactive software, such as computer games, employ complex user interfaces. Although these user interfaces make the games attractive and powerful, unfortunately they also make them extremely difficult to test. Not only do we have to deal with their functional complexity, but also the fine grained interactivity of their user interface blows up their interaction space, so that traditional automated testing techniques have trouble handling it. An agent-based testing approach offers an alternative solution: agents’ goal driven planning, adaptivity, and reasoning ability can provide an extra edge towards effective navigation in complex interaction space. This paper presents aplib, a Java library for programming intelligent test agents, featuring novel tactical programming as an abstract way to exert control over agents’ underlying reasoning-based behavior. This type of control is suitable for programming testing tasks. Aplib is implemented in such a way to provide the fluency of a Domain Specific Language (DSL). Its embedded DSL approach also means that aplib programmers will get al.l the advantages that Java programmers get: rich language features and a whole array of development tools .

Anincreasing numberofinformationsystemsarebasedonmachinelearning(ML)orartificialintelligence (AI). In some cases, the systems are adapting their behaviour during operation based on the data being gathered. This introduces new challenges for verification, validation and software testing. The traditional way of testing the systems during the development and before the deployment does not suffice anymore. IVVES (Industrial-Grade Verification and Validation of Evolving Systems) project aims to address these challenges by researching and developing methods to test ML and AI solutions and evolving systems, and using AI and ML to improve and automate development and testing. We summarise the results of the project at a high level, and provide more details on the research and collaboration related to scriptless end-to-end testing through graphical user interface.

Testing software is very important, but not done well, resulting in problematic and erroneous software applications. The cause radicates from a skills mismatch between what is needed in industry, the learning needs of students, and the way testing is currently being taught at higher and vocational education institutes. The goal of this project is to identify and design seamless teaching materials for testing that are aligned with industry and learning needs. To represent the entire socio-economic environment that will benefit from the results, this project consortium is composed of a diverse set of partners ranging from universities to small enterprises. The project starts with research in sensemaking and cognitive models when doing and learning testing. Moreover, a study will be done to identify the needs of industry for training and knowledge transfer processes for testing. Based on the outcomes of this research and the study, we will design and develop capsules on teaching software testing including the instructional materials that take into account the cognitive models of students and the industry needs. Finally, we will validate these teaching testing capsules developed during the project.

Testing software is very important, but not done well, resulting in problematic and erroneous software applications. The cause radicates from a skills mismatch between what is needed in industry, the learning needs of students, and the way testing is currently being taught at higher and vocational education institutes. The goal of this project is to identify and design seamless teaching materials for testing that are aligned with industry and learning needs. To represent the entire socio-economic environment that will benefit from the results, this project consortium is composed of a diverse set of partners ranging from universities to small enterprises. The project starts with research in sensemaking and cognitive models when doing and learning testing. Moreover, a study will be done to identify the needs of industry for training and knowledge transfer processes for testing. Based on the outcomes of this research and the study, we will design and develop capsules on teaching software testing including the instructional materials that take into account the cognitive models of students and the industry needs. Finally, we will validate these teaching testing capsules developed during the project.

Software testing is an important part of engineering trustworthyinformation systems. End-to-end testing through Graphical UserInterface (GUI) can be done manually, but it is a very time consumingand costly process. There are tools to capture or manually de ne scriptsfor automating regression testing through a GUI, but the main challengeis the high maintenance cost of the scripts when the GUI changes. Inaddition, GUIs tend to have a large state space, so creating scripts tocover all the possible paths and defining test oracles to check all the elementsof all the states would be an enormous effort. This paper presentsan approach to automatically explore a GUI while inferring state modelsthat are used for action selection in run-time GUI test generation,implemented as an extension to the open source TESTAR tool. As aninitial validation, we experiment on the impact of using various stateabstraction mechanisms on the model inference and the performance ofthe implemented action selection algorithm based on the inferred model.Later, we analyse the challenges and provide future research directionson model inference and scriptless GUI testing.

[EN] This industry showcase presents experiences on application of testar, an open source tool for scriptless testing through graphical user interface (GUI), to the web applications of Kuveyt T¿urk Participation Bank in Turkey. Kuveyt T¿urk Bank uses Selenium and Appium for regression testing of mobile and internet banking, but the maintenance cost of the test scripts is increasing day by day. Therefore, scriptless GUI testing with testar was evaluated. To provide better support for testing web-based applications, testar was extended with Selenium WebDriver integration, JavaScript support, and other new features. Results show that testar detects GUI elements much better after the improvements, and it was able to find 2 relevant errors that were not identified by existing scripted test cases., This work has been partially funded by ITEA3 TESTOMAT Project, ITEA3 IVVES project7 and EU H2020 DECODER projec

[EN] Testing is of paramount importance in assuring the quality of software products. Nevertheless, it is not easy to judge which techniques or tools are the most effective. A commonly used surrogate metric to evaluate the effectiveness of testing tools is code coverage, which has been widely used for unit and integration testing. However, for GUI testing approaches, this metric has not been sufficiently investigated. To fill this gap, we run experiments with the TESTAR tool, a scriptless testing tool that automatically generates test cases at the Graphical User Interface (GUI) level. In the experiment, we analyze and compare the obtained code coverage when using four different action selection mechanisms (ASMs) in TESTAR that are used to test three SUTs, This research has been funded by the following projects: H2020 EU project DECODER (www.decoder-project.eu), H2020 EU project iv4XR (www.iv4xrproject.eu) and ITEA project IVVES (www.ivves.eu).

Testing for quality assurance (QA) is a crucial step in the development of Extended Reality (XR) systems that typically follow iterative design and development cycles. Bringing automation to these testing procedures will increase the productivity of XR developers. However, given the complexity of the XR environments and the User Experience (UX) demands, achieving this is highly challenging. We propose to address this issue through the creation of autonomous cognitive test agents that will have the ability to cope with the complexity of the interaction space by intelligently explore the most prominent interactions given a test goal and support the assessment of affective properties of the UX by playing the role of users.

“Extended Reality” (XR) systems are advanced interactive systems such as Vir tual Reality (VR) and Augmented Reality (AR) systems. They have emerged in various domains, ranging from entertainment, cultural heritage, to combat train ing and mission critical applications. As the complexity of these systems keeps increasing, testing is getting more complex too. Current toolsets do no propose XR testing technology beyond rudimentary record and replay tools that only work for simple test scenarios. The following challenges need to be addressed: 1)Fine-grained interaction space. and 2)Assessing user experience (UX).

Modern interactive software, such as computer games, employ complex user interfaces. Although these user interfaces make the games attractive and powerful, unfortunately they also make them extremely difficult to test. Not only do we have to deal with their functional complexity, but also the fine grained interactivity of their user interface blows up their interaction space, so that traditional automated testing techniques have trouble handling it. An agent-based testing approach offers an alternative solution: agents’ goal driven planning, adaptivity, and reasoning ability can provide an extra edge towards effective navigation in complex interaction space. This paper presents aplib, a Java library for programming intelligent test agents, featuring novel tactical programming as an abstract way to exert control over agents’ underlying reasoning-based behavior. This type of control is suitable for programming testing tasks. Aplib is implemented in such a way to provide the fluency of a Domain Specific Language (DSL). Its embedded DSL approach also means that aplib programmers will get all the advantages that Java programmers get: rich language features and a whole array of development tools.

This work has been developed with the financial support of the European Union's Horizon 2020 research and innovation programme under grant agreement No. 824231

Software Engineering courses play an important role for preparing students with the right knowledge and attitude for software development in practice. The implication is far reaching, as the quality of the software that we use ultimately depends on the quality of the people that make them. Educating Software Engineering, however, is quite challenging, as the subject is not considered as most exciting by students, while teachers often have to deal with exploding number of students. The EU project IMPRESS seeks to explore the use of gamification in educating software engineering at the university level to improve students' engagement and hence their appreciation for the taught subjects. This paper presents the project, its objectives, and its current progress.

Software Engineering courses play an important role for preparing students with the right knowledge and attitude for software development in practice. The implication is far reaching, as the quality of the software that we use ultimately depends on the quality of the people that make them. Educating Software Engineering, however, is quite challenging, as the subject is not considered as most exciting by students, while teachers often have to deal with exploding number of students. The EU project IMPRESS seeks to explore the use of gamification in educating software engineering at the university level to improve students' engagement and hence their appreciation for the taught subjects. This paper presents the project, its objectives, and its current progress.

To manage the complexity of today's information systems, we need to investigate novel approaches for automated testing. In this paper we present two extensions to TESTAR, a state-of-the-art tool for testing systems through the GUI. We extend this tool with 1) a systematic and powerful approach for storing and querying test results through a graph database for offline oracles, and 2) support for accessibility evaluation of general applications for stakeholders with disabilities utilizing offline oracles. Furthermore, we conduct a preliminary validation of these extensions through a case study on the popular VLC Media Player.

Piloting an academic test automation tool in an industrialenvironment provides valuable feedback on practical applicability of thetool, but often requires a significant collaboration effort and an iterativeprocess of feedback, development, and evaluation. In this paper, we propose an approach for continuous delivery of TESTAR open source testautomation tool into an industrial continuous integration (CI) environment for piloting and evaluation.

Piloting an academic test automation tool in an industrial environment provides valuable feedback on practical applicability of the tool, but often requires a significant collaboration effort and an iterative process of feedback, development, and evaluation. In this paper, we propose an approach for continuous delivery of TESTAR open source test automation tool into an industrial continuous integration (CI) environment for piloting and evaluation.

We will show how to set up TESTAR for your own System Under Test.

This video explains how to get started with the TESTAR tool.

We will explain what TESTAR can do for you out of the box.

We will discuss what Graphical User Interfaces (GUI) are and how they are composed of widgets. We will define GUI states, widget trees and how user actions induce state changes. It will be explained what GUI test suites are composed of and why it is needed to automate the execution of these test suites.

The Unified Modelling Language (UML) is widely used for modelling software systems and its integration with executable languages, such as the Action Language for Foundational UML (ALF), provides a bridge between the graphical specification techniques used by mainstream software engineers and the precise analysis and validation techniques essential for the model-driven development of information systems. As far as we know, the idea of transforming Conceptual Schemas (CS) based on UML Class Diagrams into ALF to execute systematic ALF-based test cases against these CSs and to report defects by checking logs has not been explored to date. In this paper, we use ALF to create a testing environment to validate requirements and verify some system properties at the CS level. We also report on some of the implementation details and design decisions of our proof-of-concept tool, as well as its limitations and possible use scenarios.

Tools that perform automated software testing via the user interface rely on an action selection mechanism that at each step of the testing process decides what to do next. This mechanism is often based on random choice, a practice commonly referred to as monkey testing. In this work we evaluate a first approach to genetic programming (GP) for action selection that involves evolving IF-THEN-ELSE rules; we carry out experiments and compare the results with those obtained by random selection and also by \(\mathcal {Q}\)-learning, a reinforcement learning technique. Three applications are used as Software Under Test (SUT) in the experiments, two of which are proprietary desktop applications and the other one an open source web-based application. Statistical analysis is used to compare the three action selection techniques on the three SUTs; for this, a number of metrics are used that are valid even under the assumption that access to the source code is not available and testing is only possible via the GUI. Even at this preliminary stage, the analysis shows the potential of GP to evolve action selection mechanisms.

Existing software analysis methods for performanceand coverage are typically tied to the source code of software applications.In this work, we extend these methods to the Graphical User Interfaces (GUI) of applications, motivated by the desire to bring the user perspective into focus of software quality assurance and testing at the GUI level. We present and discuss various profiling procedures, their advantages and disadvantages, the arising challenges and the identified solutions. The identification and classification of the GUI components recorded during the monitoring process posed particular problems. The monitoring and collection of data could be well implemented, while detailed improvements in the evaluation of results are still necessary.

Part 4: Short Contributions; International audience; Test$$^*$$∗ is a testing tool that automatically and dynamically generates, executes and verifies test sequences based on a tree model that is derived from the software User Interface through assistive technologies. Test$$^*$$∗ is an academic prototype that we continuously try to transfer to companies to get feedback about its applicability. In this paper we report on one of these short experiences of using Test$$^*$$∗ in industry at the Valencian company Indenova. We applied the tool to check the localisation quality of a secure web platform that encapsulates a set of applications as services.

The innovation alliance training course is to strengthen the knowledge triangle between universities, SMEs and innovation support organizations., You can use this learning object to get to know the innovation alliance training course for strengthening the knowledge triangle between universities, SMEs and innovation support organizations.

Increasing and more ubiquitous use of mobile and Web applications with graphical user interfaces (GUIs) places more stringent requirements on the software’s quality. Automated testing is used to ensure the quality but testing through the software’s GUI is challenging and therefore a research topic that has grown during the last decade. However, despite of the evolution of automated GUI-based testing methods and tools, its industrial adoption has been limited. In this paper, we present a synthesis of the evolution of GUI-based test automation and propose a classification for methods and tools for automated regression through the GUI.

Increasing and more ubiquitous use of mobile and Web applications with graphical user interfaces (GUIs) places more stringent requirements on the software’s quality. Automated testing is used to ensure the quality but testing through the software’s GUI is challenging and therefore a research topic that has grown during the last decade. However, despite of the evolution of automated GUI-based testing methods and tools, its industrial adoption has been limited. In this paper, we present a synthesis of the evolution of GUI-based test automation and propose a classification for methods and tools for automated regression through the GUI.

Although mutation testing is a well-known technique for assessing the quality of tests, there is not a lot of support available for model-level mutation analysis. It is also considered to be expensive due to: (i) the large number of mutants generated; ii) the time-consuming activity of determining equivalent mutants; and (iii) the mutant execution time. It should also be remembered that real software artefacts of appropriate size including real faults are hard to find and prepare appropriately. In this paper we propose a mutation tool to generate valid First Order Mutants (FOM) for Conceptual Schemas (CS) based on UML Class Diagrams and evaluate its effectiveness and efficiency in generating valid and non-equivalent mutants. Our main findings were: 1) FOM mutation operators can be automated to avoiding non-valid mutants (49.1%). 2) Fewer equivalent mutants were generated (7.2%) and 74.3% were reduced by analysing the CS static structure in six subject CSs.

Although mutation testing is a well-known technique for assessing the quality of tests, there is not a lot of support available for model-level mutation analysis. It is also considered to be expensive due to: (i) the large number of mutants generated; ii) the time-consuming activity of determining equivalent mutants; and (iii) the mutant execution time. It should also be remembered that real software artefacts of appropriate size including real faults are hard to find and prepare appropriately. In this paper we propose a mutation tool to generate valid First Order Mutants (FOM) for Conceptual Schemas (CS) based on UML Class Diagrams and evaluate its effectiveness and efficiency in generating valid and non-equivalent mutants. Our main findings were: 1) FOM mutation operators can be automated to avoiding non-valid mutants (49.1%). 2) Fewer equivalent mutants were generated (7.2%) and 74.3% were reduced by analysing the CS static structure in six subject CSs.

Usability is currently a key feature for developing quality systems. A system that satisfies all the functional requirements can be strongly rejected by end-users if it presents usability problems. End-users demand intuitive interfaces and an easy interaction in order to simplify their work. The first step in developing usable systems is to determine whether a system is or is not usable. To do this, there are several proposals for measuring the system usability. Most of these proposals are focused on the final system and require a large amount of resources to perform the evaluation (end-users, video cameras, questionnaires, etc.). Usability problems that are detected once the system has been developed involve a lot of reworking by the analyst since these changes can affect the analysis, design, and implementation phases. This paper proposes a method to minimize the resources needed for the evaluation and reworking of usability problems. We propose an early usability evaluation that is based on conceptual models. The analyst can measure the usability of attributes that depend on conceptual primitives. This evaluation can be automated taking as input the conceptual models that represent the system abstractly.

TESTAR, a tool for test automation at the user interface level

how to use the e-learning course of the Innovative Trainer workshop.

Este vídeo presente 3 pasos que puede seguir el alumno para hacer definiciones recursivas. Los 3 pasos están explicados a través de 2 ejemplos usando Java y Haskell. Para similar este objeto de aprendizaje, el alumno debería practicas haciendo definiciones recursivas después de haber visto el video

This is a video that presents a high level overview of the FITTEST project for Future Internet Testing. What is it, Who is participating, Why is it needed, What is delivered, Who can benefit and How can you participate.