Search

Context: Software testing is the most used technique for quality assurance in industry. However, in computer science education software testing is still treated as a second-class citizen and students are unable to test their software well enough. One reason for this is that teaching the subject of software testing is difficult as it is a complex intellectual activity for which students need to allocate multiple cognitive resources at the same time. A myriad of primary and secondary studies have tried to solve this problem in education, however still with very limited results. Objective: Before we can design interventions to improve our pedagogical approaches, we need to gain more in-depth understanding and recognition of sensemaking as it is happening when students design test cases. Method: An initial exploratory study identified four different sensemaking approaches used by students while creating test models. In this paper we present a follow-up study with 50 students from a large university in Spain. The used methodology was based on the previous study with the improvements that originated from its evaluation. We asked the participants to create a test model based on a description of a test problem using a specialized web-based tool for modeling test cases. We measured how well these models fit the test problem, the sensemaking process that students went through when creating the models, and the students’ perception of the modeling task. The participants received no compensation for their efforts, and we scheduled the experiment during a regular class. Apart from the created models and their metadata, we also collected recordings of the students’ computer screens made during the experiment and used a questionnaire to study their perspectives on the assignment. All the collected textual, graphical, and video data was analyzed using an iterative inductive analysis process to allow new information about the different sensemaking approaches to emerge. Results: We gained better insights into the sensemaking processes of students while modeling test cases for a problem. The results enabled us to refine our previous findings, and we identified new sensemaking approaches. Conclusions: Based on these results, we can further investigate ways to influence the sensemaking process in education, the possible misconceptions that have a negative influence on it, and the desired mental model we want our students to have to design test cases.

Software Engineering courses play an important role for preparing students with the right knowledge and attitude for software development in practice. The implication is far reaching, as the quality of the software that we use ultimately depends on the quality of the people that make them. Educating Software Engineering, however, is quite challenging, as the subject is not considered as most exciting by students, while teachers often have to deal with exploding number of students. The EU project IMPRESS seeks to explore the use of gamification in educating software engineering at the university level to improve students' engagement and hence their appreciation for the taught subjects. This paper presents the project, its objectives, and its current progress.

To manage the complexity of today's information systems, we need to investigate novel approaches for automated testing. In this paper we present two extensions to TESTAR, a state-of-the-art tool for testing systems through the GUI. We extend this tool with 1) a systematic and powerful approach for storing and querying test results through a graph database for offline oracles, and 2) support for accessibility evaluation of general applications for stakeholders with disabilities utilizing offline oracles. Furthermore, we conduct a preliminary validation of these extensions through a case study on the popular VLC Media Player.

Existing software analysis methods for performanceand coverage are typically tied to the source code of software applications.In this work, we extend these methods to the Graphical User Interfaces (GUI) of applications, motivated by the desire to bring the user perspective into focus of software quality assurance and testing at the GUI level. We present and discuss various profiling procedures, their advantages and disadvantages, the arising challenges and the identified solutions. The identification and classification of the GUI components recorded during the monitoring process posed particular problems. The monitoring and collection of data could be well implemented, while detailed improvements in the evaluation of results are still necessary.

Part 4: Short Contributions; International audience; Test$$^*$$∗ is a testing tool that automatically and dynamically generates, executes and verifies test sequences based on a tree model that is derived from the software User Interface through assistive technologies. Test$$^*$$∗ is an academic prototype that we continuously try to transfer to companies to get feedback about its applicability. In this paper we report on one of these short experiences of using Test$$^*$$∗ in industry at the Valencian company Indenova. We applied the tool to check the localisation quality of a secure web platform that encapsulates a set of applications as services.

Usability is currently a key feature for developing quality systems. A system that satisfies all the functional requirements can be strongly rejected by end-users if it presents usability problems. End-users demand intuitive interfaces and an easy interaction in order to simplify their work. The first step in developing usable systems is to determine whether a system is or is not usable. To do this, there are several proposals for measuring the system usability. Most of these proposals are focused on the final system and require a large amount of resources to perform the evaluation (end-users, video cameras, questionnaires, etc.). Usability problems that are detected once the system has been developed involve a lot of reworking by the analyst since these changes can affect the analysis, design, and implementation phases. This paper proposes a method to minimize the resources needed for the evaluation and reworking of usability problems. We propose an early usability evaluation that is based on conceptual models. The analyst can measure the usability of attributes that depend on conceptual primitives. This evaluation can be automated taking as input the conceptual models that represent the system abstractly.

Part 1: Long and Short Papers; International audience; According to the Model-Driven Development (MDD) paradigm, analysts can substantially improve the software development process concentrating their efforts on a conceptual model, which can be transformed into code by means of transformation rules applied by a model compiler. However, MDD tools are not widely used in industry. One of the reasons for this poor adoption is the lack of usability of MDD tools. This paper presents a framework to evaluate the usability of such tools. The framework will be used as a basis for a family of experiments to get clear insights into the barriers to usability that prevent MDD tools from being widely adopted in industry.

Despite much scepticism and problems for its adoption, the Model-Driven Development (MDD) is being used and improved to provide many inherent benefits for industry. One of its greatest benefits is the ability to handle the complexity of software development by raising the abstraction level. Models are expressed using concepts that are not related to a specific implementation technology (e.g. Unified Modelling Language -UML, Object Constraint Language -OCL, Action Language for Foundational UML -ALF), which means that the models can be easier to specify, maintain and document. As in Model-Driven Engineering (MDE), the primary artefacts are the conceptual models, efforts are focused on their creation, testing and evolution at different levels of abstraction through transformations because if a conceptual schema has defects, these are passed on to the following stages, including coding. Thus, one of the challenges for researchers and developers in Model-Driven Development is being able to identify defects early on, at the conceptual schema level, as this helps reduce development costs and improve software quality. Over the last decade, little research work has been performed in this area. Some of the causes of this are the high theoretical complexity of testing conceptual schemas and the lack of adequate software support. This research area thus admits new methods and techniques, facing challenges such as generation of test cases using information external to the conceptual schemas (i.e. requirements), the measurement of possible automation, selection and prioritization of test cases, the need for an efficient support tool using standard semantics, the opportune feedback to support the software quality assurance process and facilitate making decisions based on the analysis and interpretation of the results.The aim of this thesis is to mitigate some of the problems that affect conceptual schema validation by providing a novel testing-based validation framework based on Model-Driven Development. The use of MDD improves abstraction, automation and reuse, which allows us to alleviate the complexity of our validation framework. Furthermore, by leveraging MDD techniques (such as metamodeling, model transformations, and models at runtime), our framework supports four phases of the testing process: test design, test case generation, test case execution and the evaluation of the results.In order to provide software support for our proposal, we developed the CoSTest ALF-based testing environment. To ensure that CoSTest offers the necessary functionality, we first identified a set of functional requirements. Then, after these requirements were identified, we defined the architecture and testing environment of the validation framework, and finally we implemented the architecture in the Eclipse context. CoSTest has been developed to test several properties on the executable model, such as syntactic correctness (i.e. all the elements in the model conform to the syntax of the language in which it is described), consistency between the structural and behavioural parts (its integrity constraints) and completeness (i.e. all possible changes on the system state can be performed through the execution of the operations defined in the executable model). For defective models, the CoSTest report returns a meaningful feedback that helps locate and repair any defects detected., A pesar del escepticismo y dificultades en su adopción, el Desarrollo Orientado por Modelos (MDD, por sus siglas en inglés) está siendo usado y mejorado para proveer muchos beneficios inherentes a la industria. Uno de sus mayores beneficios es la capacidad de manejar la complejidad del desarrollo de software elevando el nivel de abstracción. Los modelos se expresan utilizando conceptos que no están relacionados con una tecnología de implementación específica (por ejemplo, Lenguaje de Modelado Unificado -UML, Lenguaje de Restricción de Objetos -OCL, Lenguaje de Acción para el Foundational UML - ALF), lo que significa que los modelos pueden ser más fáciles de especificar, mantener y documentar. Debido a que en una Ingeniería dirigida por modelos (MDE), los artefactos primarios son los modelos conceptuales, los esfuerzos se centran en su creación, prueba y evolución a diferentes niveles de abstracción a través de transformaciones, porque si un esquema conceptual tiene defectos, éstos se pasan a las siguientes etapas, incluida la codificación. Por lo tanto, uno de los retos para los investigadores y desarrolladores in MDD es poder identificar los defectos temprano, a nivel de esquemas conceptuales, ya que esto ayudaría a reducir los costos de desarrollo y mejorar la calidad del software.Durante la última década, pocos trabajos de investigación se han realizado en esta área. Algunas de las causas de esta realidad son la alta complejidad teórica de probar esquemas conceptuales y la falta de soporte de software adecuado. Por lo tanto, este área de investigación admite nuevos métodos y técnicas, enfrentando retos como la generación de casos de prueba utilizando información externa a los esquemas conceptuales (es decir, los requisitos), la medición de una posible automatización, selección y priorización de casos de prueba, la necesidad de una herramienta de soporte eficiente que utilice una semántica estándar, la retroalimentación oportuna para apoyar el proceso de aseguramiento de la calidad del software y facilitar la toma de decisiones basadas en el análisis y la interpretación de los resultados.El objetivo de esta tesis es mitigar algunos de los problemas que afectan la validación de los esquemas conceptuales, proporcionando un nuevo marco de validación basado en pruebas que fue construido usando un desarrollo dirigido por modelos. El uso de MDD permite un aumento en la abstracción, automatización y reutilización que nos permite aliviar la complejidad de nuestro marco de validación. Además, al aprovechar las técnicas MDD (como el metamodelado, las transformaciones de modelos y los modelos en tiempo de ejecución), nuestro marco soporta cuatro fases del proceso de prueba: diseño de pruebas, generación de casos de prueba, ejecución de casos de prueba y la evaluación de los resultados.Con el fin de proporcionar soporte de software para nuestra propuesta, hemos desarrollado CoSTest, un entorno de pruebas basado en el lenguaje ALF. Para asegurar que CoSTest ofrece la funcionalidad necesaria, primero identificamos un conjunto de requisitos funcionales. Luego, después de identificar estos requisitos, definimos la arquitectura y el ambiente de pruebas de nuestro marco de validación y, finalmente, implementamos la arquitectura en el contexto de Eclipse. CoSTest ha sido desarrollado para probar varias propiedades sobre el modelo ejecutable como la corrección sintáctica (es decir, todos los elementos del modelo se ajustan a la sintaxis del lenguaje en el que se describe), consistencia entre la parte estructural y el comportamiento (sus restricciones de integridad) y completitud (es decir, todos los cambios posibles en el estado del sistema se pueden realizar a través de la ejecución de las operaciones definidas en el modelo ejecutable). Para los modelos defectuosos, el informe de CoSTest devuelve una retroalimentación significativa que ayuda a localizar y reparar los defectos detec, A pesar de l'escepticisme i les dificultats en la seua adopció, el Desenvolupament Orientat per Models (MDD, segons les sigles en anglès) està sent usat i millorat per tal de proveir molts beneficis potencials inherents a l' indústria. Un dels majors beneficis és la capacitat de manejar la complexitat del desenvolupament del programari elevant el nivell d'abstracció. Els models s'expressen mitjançant conceptes que no estan relacionats amb una tecnologia d'implementació específica (per exemple, el Llenguatge de Modelat Unificat - UML, Llenguatge de Restricció d'Objectes -OCL, Llenguatge d'Acció per al Foundational UML - ALF), el que significa que els models poder ser més fàcils d'especificar, mantindre i documentar. A causa de que en una Enginyeria dirigida per models (MDE), els artefactes primaris són els models conceptuals, els esforços es centren en la seua creació, prova i evolució a diferents nivells d'abstracció mitjançant transformacions, perquè si un esquema conceptual té defectes, aquestos es passen a les següents etapes, inclosa la codificació. Per tant, un del reptes per als investigadors i desenvolupadors en MDD és poder identificar els defectes des del principi, a nivell de esquemes conceptuals, perquè açò ajudaria a reduir els costos de desenvolupament i millora de la qualitat del programari.Durant l'última dècada, pocs treballs d'investigació s'han fet en aquesta àrea. Algunes de les causes d'aquesta realitat són l'alta complexitat teòrica de provar esquemes conceptuals i la falta de suport de programari adequat. Per tant, aquesta àrea d'investigació admet nous mètodes i tècniques, enfrontant reptes com la generació de casos de prova mitjançant informació externa als esquemes conceptuals (es a dir, requisits), la medició de una possible automatització, selecció i priorització de casos de prova, la necessitat de una ferramenta de suport rentable que utilitze una semàntica estàndard, la retroalimentació oportuna per suportar el procés d'assegurament de la qualitat del programari i la facilitat per a prendre decisions basades en l'anàlisi i la interpretació dels resultats.En aquesta tesi intentem mitigar alguns dels problemes que afecten a la validació dels esquemes conceptuals, proporcionant un nou marc de validació basat en proves que va ser construït mitjançant un desenvolupament dirigit per models. L'ús de MDD permet un augment en l'abstracció, automatització i reutilització que ens permet alleujar la complexitat del nostre marc de validació. A més a més, al aprofitar les tècniques MDD (com el metamodelat, les transformacions de models i els models en temps d'execució), el nostre marc suporta quatre fases del procés de prova: disseny, generació i execució de casos de prova, així com l'avaluació de resultats del procés de prova. Amb la finalitat de proporcionar suport de programari per a la nostra proposta, hem desenvolupat un entorn de proves basat en el llenguatge ALF que s'anomena CoSTest. Per tal d'assegurar que CoSTest ofereix la funcionalitat necessària, identifiquem un conjunt de requisits funcionals abans de desenvolupar la ferramenta. Després d'identificar aquestos requisits, definim l'arquitectura i l'ambient de proves del nostre marc de validació, i finalment, implementem l'arquitectura en el context Eclipse. CoSTest ha sigut desenvolupat per provar diverses propietats sobre el model executable com la correcció sintàctica (és a dir, tots els elements del model s'ajusten a la sintaxi del llenguatge en el que es descriu), consistència antre la part estructural i el comportament (les seues restriccions d'integritat) i completitud (és a dir, tots els canvis possibles en l'estat del sistema es poden realitzar mitjançant l'execució de les operacions definides en el model executable). Per als models defectuosos, l'informe de CoSTest retorna una retroalimentació significativa que ajuda a localitzar i reparar els defectes dete, Granda Juca, MF. (2017). Testing-Based Conceptual Schema Validation in a Model-Driven Environment [Tesis doctoral no publicada]. Universitat Politècnica de València. doi:10.4995/Thesis/10251/89091., TESIS