Search

Interactive Adaptive Systems (IAS) refer to interactive systems that alter their characteristics automatically according to the user’s needs. The IAS have emerged in several application domains and, accordingly, a high level of quality must be assured during their development. However, evaluating IAS still presents different challenges, as there are a set of unique characteristics that can impact the quality of such systems. Moreover, there does not exist a specific study to guide the assessment of IAS considering their main challenges based on an international standard. The main objective of this research concerns the proposition of an evaluation model suitable to the context of interactive adaptive systems, based on ISO/IEC 25040 standard. In the case study, an adaptive hypermedia system is evaluated following the evaluation process restructured from ISO/IEC 25040, with a focus on the first three activities of this process.

The choice of suitable evaluation methods for the layered evaluation of Interactive Adaptive Systems (IAS) needs the consideration of different factors, leading to a multi-criteria decision analysis problem. This paper proposes the first step toward a Web-based Multi-Criteria Decision Support System (MCDSS). Our proposal is based on a recent multi-criteria decision method called ELECTRE TRI-B-H (Elimination and Choice Translating Reality) to guide the layered evaluation of IAS. The goal is to support the choice of appropriate evaluation methods for individual layers taking into account constraints of the layered evaluation and the individual layers. The appropriateness of each evaluation method is analyzed for the layered evaluation in general and each layer in particular. The use of ELECTRE TRI-B-H method allows decomposing a decision problem into intermediate sub-problems through a hierarchy model and sorting alternative evaluation methods at different levels of the hierarchy. A case study of an adaptive hypermedia system is presented; in this study, the ELECTRE TRI-B-H method has been applied to select the most suitable evaluation methods for each of its layers taking into account the layered evaluation context. The promising results show the feasibility of the proposed approach, leading to various research perspectives.

Cybersickness (CS) is an affliction that limits the use of virtual reality (VR) applications. For decades, the measurement of cybersickness has presented one of the most challenges that have aroused the interest of VR research community. Having strong effects on users’ health, cybersickness causes several symptoms relating to different factors. In most cases, the literature studies for VR cybersickness evaluation adopt the questionnaire-based approaches. Some studies have focused on physiological and postural instability-based approaches, while others support the VR content. Despite the attention paid to define measurements for assessing cybersickness, there is still a need for a more complete evaluation model that allows measuring cybersickness in real time. This paper defines a conceptual model that integrates subjective and objective evaluation of CS in real time. The proposed model considers three CS factors (i.e. individual, software and hardware). The aim is to consider the heterogeneous findings (subjective and objective measures) related to the selected CS factors that define integrated indicators. The theoretical part of the model was initially validated by researchers who have comprehensive knowledge and skills in VR domain. As a research perspective, we intend to evaluate the proposed model through a practical case study.

Successful usability evaluation of interactive adaptive systems requires the use of appropriate methods. Different usability evaluation methods exist in the literature. These methods can be applied in association with the individual layers and with the whole adaptive system. In each situation, it is essential to apply the appropriate evaluation methods. In this paper, we propose a novel decision support approach called EvalCHOICE. It guides evaluators in the choice of appropriate usability evaluation methods for individual layers and the adaptive system as a whole. Our proposal is applied to determine the suitable evaluation methods for a given Web-based adaptive system in the field of transport.

Part 5: Usability Evaluation and UI Testing; International audience; Choosing the appropriate usability evaluation methods is a key part of the usability evaluation process of interactive adaptive systems. This step needs the consideration of different factors, leading to a multi-criteria decision analysis problem. In this paper, we present a review of the main factors reported in the literature which can affect the selection of usability evaluation methods for interactive adaptive systems. Three of the most commonly used usability evaluation methods are selected and classified according to these factors. The results of this research are used by applying a decision aid method in order to guide the choice of suitable usability evaluation methods for a given adaptive system in the field of tourism.

Many usability evaluation methods for interactive adaptive systems exist in the literature. There is not yet an agreement in the adaptive system community about which method is more useful than another in specific evaluation situations. This raises the question, “What is (are) the best evaluation method(s) that need(s) to be used in specific evaluation constraints?” This paper presents possible directions to address this issue by proposing a multi-criteria decision support framework for selecting the appropriate evaluation methods for interactive adaptive systems. The proposed decision support framework is applied to determine the suitable usability evaluation methods for a specific adaptation layer of a given adaptive hypermedia system.

International audience; The layered evaluation of interactive adaptive systems has to consider many evaluation methods. The best evaluation method to be used for individual layers depend on many parameters such as the evaluation criteria, the stage of the development cycle, and the characteristics of the layer under consideration. This paper presents a decision model for selecting the appropriate evaluation methods for individual layers of the interactive adaptive system. Our proposal is based on one multi-criteria method, namely ELECTRE TRI method. The proposed decision model is applied to determine the suitable evaluation methods for an adaptive hypermedia system.

International audience

Usability is a crucial factor for interactive adaptive system success. Several methods have been proposed to support the usability evaluation of interactive adaptive systems. However, the diversity of these methods leaves the evaluators wondering what would be the most appropriate usability evaluation methods for their specific goals. This paper examines the characteristics of usability evaluation methods for interactive adaptive systems and identifies the most relevant features for categorizing them. The comparison and classification of these methods will serve as a reference for researchers in the field of interactive adaptive systems and as a guide for the choice of suitable usability evaluation methods.

The paper presents a new tool for the automatic evaluation of the usability of interactive systems (WIMP and Web). The proposed tool is based on a set of Web services, each of which service uses a set of ergonomic guidelines to evaluate particular graphical controls. One main advantage of the proposed tool is that it provides for flexible management of ergonomic guidelines independently of the evaluation engine. We illustrate the efficiency of our tool through an example of an evaluation support system.

International audience; The evaluation of interactive adaptive systems is considered as a difficult endeavour due to the complex nature of these systems. Issues arise in the need to identify, for a given evaluation situation the set of suitable usability evaluation methods to be used by the evaluators. This paper describes a framework for supporting the choice of appropriate methods for the usability evaluation of interactive adaptive systems. It is based on the comparison of usability evaluation methods using relevant criteria. This framework is then validated through a case study; it concerns an adaptive hypermedia system in transportation field.

The evaluation of interactive adaptive systems is the source of many difficulties for novice evaluators as well as for expert ones. Issues arise in the need to identify the appropriate evaluation methods to be used for a given evaluation context. This paper proposes an approach for the choice of evaluation methods for interactive adaptive systems with regard to their suitability depending on specific evaluation contexts. A multicriteria decision making approach, namely the Analytic Hierarchy Process, is proposed for the selection methodology. The approach is based on a pair-wise comparison of a number of criteria that affect the choice of the appropriate evaluation methods. Four user-centred evaluation methods in conjunction with the layered evaluation for interactive adaptive systems are presented in the cases studies. These methods are: user-as-wizard, heuristic evaluation, focus group, and user test. The proposed approach is flexible in the sense that it can be used in the case of different evaluation methods. The results depend on the evaluation context where the methods will be used. Hence, we illustrate the proposed approach in the case of two evaluation contexts.