Search

Software testing is an important part of engineering trustworthyinformation systems. End-to-end testing through Graphical UserInterface (GUI) can be done manually, but it is a very time consumingand costly process. There are tools to capture or manually de ne scriptsfor automating regression testing through a GUI, but the main challengeis the high maintenance cost of the scripts when the GUI changes. Inaddition, GUIs tend to have a large state space, so creating scripts tocover all the possible paths and defining test oracles to check all the elementsof all the states would be an enormous effort. This paper presentsan approach to automatically explore a GUI while inferring state modelsthat are used for action selection in run-time GUI test generation,implemented as an extension to the open source TESTAR tool. As aninitial validation, we experiment on the impact of using various stateabstraction mechanisms on the model inference and the performance ofthe implemented action selection algorithm based on the inferred model.Later, we analyse the challenges and provide future research directionson model inference and scriptless GUI testing.

“Extended Reality” (XR) systems are advanced interactive systems such as Vir tual Reality (VR) and Augmented Reality (AR) systems. They have emerged in various domains, ranging from entertainment, cultural heritage, to combat train ing and mission critical applications. As the complexity of these systems keeps increasing, testing is getting more complex too. Current toolsets do no propose XR testing technology beyond rudimentary record and replay tools that only work for simple test scenarios. The following challenges need to be addressed: 1)Fine-grained interaction space. and 2)Assessing user experience (UX).

To manage the complexity of today's information systems, we need to investigate novel approaches for automated testing. In this paper we present two extensions to TESTAR, a state-of-the-art tool for testing systems through the GUI. We extend this tool with 1) a systematic and powerful approach for storing and querying test results through a graph database for offline oracles, and 2) support for accessibility evaluation of general applications for stakeholders with disabilities utilizing offline oracles. Furthermore, we conduct a preliminary validation of these extensions through a case study on the popular VLC Media Player.

Tools that perform automated software testing via the user interface rely on an action selection mechanism that at each step of the testing process decides what to do next. This mechanism is often based on random choice, a practice commonly referred to as monkey testing. In this work we evaluate a first approach to genetic programming (GP) for action selection that involves evolving IF-THEN-ELSE rules; we carry out experiments and compare the results with those obtained by random selection and also by \(\mathcal {Q}\)-learning, a reinforcement learning technique. Three applications are used as Software Under Test (SUT) in the experiments, two of which are proprietary desktop applications and the other one an open source web-based application. Statistical analysis is used to compare the three action selection techniques on the three SUTs; for this, a number of metrics are used that are valid even under the assumption that access to the source code is not available and testing is only possible via the GUI. Even at this preliminary stage, the analysis shows the potential of GP to evolve action selection mechanisms.

Usability is currently a key feature for developing quality systems. A system that satisfies all the functional requirements can be strongly rejected by end-users if it presents usability problems. End-users demand intuitive interfaces and an easy interaction in order to simplify their work. The first step in developing usable systems is to determine whether a system is or is not usable. To do this, there are several proposals for measuring the system usability. Most of these proposals are focused on the final system and require a large amount of resources to perform the evaluation (end-users, video cameras, questionnaires, etc.). Usability problems that are detected once the system has been developed involve a lot of reworking by the analyst since these changes can affect the analysis, design, and implementation phases. This paper proposes a method to minimize the resources needed for the evaluation and reworking of usability problems. We propose an early usability evaluation that is based on conceptual models. The analyst can measure the usability of attributes that depend on conceptual primitives. This evaluation can be automated taking as input the conceptual models that represent the system abstractly.