Search

This report first takes stock of XAI-related requirements appearing in various EU directives, regulations, guidelines, and CJEU case law. This analysis of existing requirements will permit us to have a clearer vision of the purposes, the ``why'', of XAI, which we separate into five categories: contestability, empowerment/redressing information asymmetries, control over system performance, evaluation of algorithmic decisions, and public administration transparency. The analysis of legal requirements also permits us to create four categories of recipients for explainability: data science teams; human operators of the system; persons affected by algorithmic decisions, and regulators/judges/auditors. Lastly, we identify four main operational contexts for explainability: XAI for the upstream design and testing phase; XAI for human-on-the-loop control; XAI for human-in-the-loop control; and XAI for ex-post challenges and investigations.Second, we will present user-centered design methodology, which takes the purposes, the recipients and the operational context into account in order to develop optimal XAI solutions.Third, we will suggest a methodology to permit suppliers and users of high-risk AI applications to propose local XAI solutions that are effective in the sense of being ``meaningful'', for example, useful in light of the operational, safety and fundamental rights contexts. The process used to develop these ``meaningful'' XAI solutions will be based on user-centric design principles examined in the second part.Fourth, we will suggest that the European Commission issue guidelines to provide a harmonised approach to defining ``meaningful'' explanations based on the purposes, audiences and operational contexts of AI systems. These guidelines would apply to the AI Act, but also to the other EU texts requiring explanations for algorithmic systems and results.

Context-aware systems are systems that adapt their behavior dynamically when sensing changes in the surrounding environment in which they run. These changes may come from changing user preferences, external sensors (weather, localisation), or internal sensors (battery, memory) of the device. The paradigm of context-oriented programming, provides dedicated programming language abstractions to adapt the behaviour of a software system dynamically upon changing contexts. In context-oriented programming (COP), contexts and behavioural adaptations are first-class language entities. The behavioural adaptations get (de)activated in the code whenever their corresponding contexts become (de)activated. Many different COP languages exist. We also proposed our own specific programming framework, RubyCOP, to implement context-oriented systems in which we explicitly separate the notions of contexts and features. This clear separation promotes a better maintainability and reusability in developping such systems. Our RubyCOP programming framework is part of a more complete approach to feature-based context-oriented software development, which also consists of an architecture, a supporting development methodology and two visualisation tools. This presentation will focus only on the programming framework.

Context-oriented programming enables dynamic software evolution by supporting the creation of software systems that dynamically adapt their behaviour depending on the context of their surrounding environment. Upon sensing a new particular situation in the surrounding environment, which is reified as a context, the system activates this context and then continues by selecting and activating fine-grained features corresponding to that context. These features, representing functionalities specific to that context, are then installed in the system to refine its behaviour at runtime. Conceiving such systems is complex due to their high dynamicity and the combinatorial explosion of possible contexts and corresponding features that could be active. To address this complexity, we propose a feature-based context-oriented software development approach to design and implement context-oriented applications. This approach unifies context-oriented programming, feature modelling and dynamic software product lines into a single paradigm. In this novel paradigm we separate clearly and explicitly contexts and features that we model in terms of a context model, a feature model and the mapping between them. We also design an architecture, implement a programming framework, and develop a supporting development methodology and two visualisation tools to help designers and programmers in their modelling, development and debugging tasks. Furthermore we also develop a user interface library in our approach to create applications with user interfaces that are adaptive. To validate our feature-based context-oriented software development approach, we designed five case studies and implemented three of them. Then we discussed the design qualities to evaluate our implementation of the programming framework. We also assessed the usability of the programming framework from our own perspective based on the cognitive dimensions of notations framework. Finally we also conducted four user studies with, (FSA - Sciences de l'ingénieur) -- UCL, 2022

With the transformation of computing from personal computers to the Internet, document formats have also seen some changes over the years. Future document formats are likely going to adapt to the emerging needs of ubiquitous computing, where information processing is embedded in everyday activities and objects. While most existing document formats have originally been a digital emulation of paper documents, over the years they have been enriched with additional digital features. These features were mainly incorporated to take advantage of the new functionality offered by the devices on which the documents are accessed. With the advent of ubiquitous computing, document formats seem to be facing the next evolutionary step. They will have to adapt to novel mobile devices, innovative interaction modalities, the distribution over multiple devices as well as heterogeneous input sources. This adaptation to the age of ubiquitous computing asks for several new document features. We outline a roadmap towards future fluid document representations for ubiquitous information environments. Based on the resource-selector-link (RSL) hypermedia metamodel - a general hypermedia metamodel supporting distribution, user rights management and content adaptation - we developed a metamodel for fluid document formats and the corresponding online text editor for fluid documents.

Context-oriented programming enables dynamic software evolution by supporting the creation of software systems that dynamically adapt their behaviour depending on the context of their surrounding environment. Conceiving such systems is complex due to their high dynamicity and the combinatorial explosion of possible contexts and corresponding features that could be active. We present the current status of our feature-based context-oriented programming approach, illustrate how it helps software developers to conceive highly-dynamic systems, and discuss some remaining challenges of our approach.

We implemented a feature-based context-oriented programming language, which clearly separates the notion of contexts from the notion of features. Contexts reify particular situations occurring in the surrounding environment, to which a software system can adapt. Features reify the system's behaviour; they are the language components that describe the system's functionality at a fine-grained level. Contexts are mapped to features, such that, when certain contexts become active at run-time, the corresponding features get selected and activated, thus adapting the system's functionality to those particular contexts. In this paper we show the object-oriented architecture, design and implementation issues of such a feature-based context-oriented programming language, which we implemented on top of the Ruby programming language as an application framework for context-oriented programmers. An important part of our language design is the explicit representation of contexts and features in terms of hierarchical tree structures that capture the structural constraints to be maintained at runtime. We illustrate our language design with a small example of a feature-based context-oriented program written in this language.

In recent years, public displays have been studied as a way to foster citizen participation. However, their surroundings and users are prone to high variability, which makes it tedious to accommodate different contexts with an optimal participation experience. In this paper, we propose adaptive public displays as a lead for solution in tackling this issue. From a review of the motivators and barriers affecting citizen's interaction with public displays we defined a process model destined to serve as a guide for designers of such systems.

In this paper we present a visualisation tool that is intricately related to a feature-based context-oriented programming language. Context-oriented programming languages allow programmers to develop software systems of which the behaviour evolves dynamically upon changing contexts. In our language , the software behaviour as well as the contexts to which the behaviour adapts, are encoded in terms of separate feature models. Due to the highly dynamic nature of such software systems and the many possible combinations of contexts to which they may adapt, developing such systems is hard. To help programmers manage the complexity of developing such software systems, we created a tool to help them visualise the contexts and features , even at runtime. The visualisation tool confronts two hierarchical models: the context model and the feature model, and highlights the dependencies between them. We conduct an initial user study of the visualisation tool to assess its usefulness and usability.

Context-oriented programming languages allow programmers to develop context-aware systems that can adapt their behaviour dynamically upon changing contexts. Due to the highly dynamic nature of such systems and the many possible combinations of contexts to which such systems may adapt, developing such systems is hard. Feature-based context-oriented programming helps tackle part of this complexity by modelling the possible contexts, and the different behavioural adaptations they can trigger, as separate feature models. Tools can also help developers address the underlying complexity of this approach. This paper presents a visualisation tool that is intricately related to the underlying architecture of a feature-based context-oriented programming language, and the context and feature models it uses. The visualisation confronts two hierarchical models (a context model and a feature model) and highlights the dependencies between them. An initial user study of the visualisation tool is performed to assess its usefulness and usability.

Euclidean distance is traditionally used to compare a gesture candidate against gesture templates in two-dimensional gesture recognizers. This paper compares two distances borrowed from other domains of computer science used in template-based two-dimensional gesture recognizers: the Mahalanobis distance, typically used in computer vision and statistics, and the Jaro-Winckler distance, typically used in information retrieval and pattern recognition. Although the geometric interpretation of these distances is less straightforward for designers, there is a significant impact of the Mahalanobis distance on recognition rate, but not for the Jaro-Winkler one.

As part of our ongoing research on context-oriented software technology, we propose a feature-oriented programming approach to context-oriented programming. Behavioural variations are implemented as fine-grained features that can be installed and activated dynamically, upon changing contexts. Given the highly dynamic nature of such a programming approach, and to cope with the complexity of many behavioural variations, that can depend on many varying contexts, developers could benefit from visual inspection tools to analyse what contexts and features are currently active, in which order they have been activated, and what code they adapt. We present a prototype of such a visualisation tool, and discuss potential improvements to that tool.

As part of our ongoing research on context-oriented software technology, we propose a feature-oriented programming approach to context-oriented programming. Behavioural variations are implemented as fine-grained features that can be installed and activated dynamically, upon changing contexts. Given the highly dynamic nature of such a programming approach, and to cope with the complexity of many behavioural variations, that can depend on many varying contexts, developers could benefit from visual inspection tools to analyse what contexts and features are currently active, in which order they have been activated, and what code they adapt. We present a prototype of such a visualisation tool, and discuss potential improvements to that tool.

This article introduces the problem of modeling multimodal interaction, in the form of markup languages. After an analysis of the current state of the art in multimodal interaction description languages, nine guidelines for languages dedicated at multimodal interaction description are introduced, as well as four different roles that such language should target: communication, configuration, teaching and modeling. The article further presents the SMUIML language, our proposed solution to improve the time synchronicity aspect while still fulfilling other guidelines. SMUIML is finally mapped to these guidelines as a way to evaluate their spectrum and to sketch future works

In this document we provide an extensive, though not exhaustive, overview of existing related research in the areas of behavioural adaptation, user interface adaptation and database adaptation.

In this presentation, we present the need for a unified vision on context-oriented software engineering, that reconciles different research areas in computer science, including programming languages, human-computer interaction, and database technology. We identify three dimensions wherein proposed solutions for achieving context orientation could benefit from cross-fertilisation and integration. We present a conceptual and implementation framework that allows for a characterisation of context-oriented systems regardless of these dimensions, serving as a vehicle to discuss some of the challenges involved in reconciling the different dimensions and how to tackle them.

This article introduces the problem of modeling multimodal interaction, in the form of markup languages. After an analysis of the current state of the art in multimodal interaction description languages, nine guidelines for languages dedicated at multimodal interaction description are introduced, as well as four different roles that such language should target: communication, configuration, teaching and modeling. The article further presents the SMUIML language, our proposed solution to improve the time synchronicity aspect while still fulfilling other guidelines. SMUIML is finally mapped to these guidelines as a way to evaluate their spectrum and to sketch future works.

Le domaine de l’interaction multimodale a pris beaucoup d’importance lors de cette dernière décennie, du fait de la création et de la mise sur le marché de nombreuses interfaces novatrices, ainsi que de l’évolution de domaines de recherche tels que la reconnaissance vocale. Cependant, la fusion et la combinaison de modalités n’ont pas évolué avec la même vigueur, aboutissant à un fossé entre l’utilisation de modalités d’entrée et les différentes possibilités de les combiner. Cette thèse de doctorat s’attache à réduire ce fossé entre les dispositifs d’interactions à disposition et la fusion de leurs données. Dans cette recherche, la fusion d’inputs multimodaux est approchée dans sa globalité : en premier lieu, sous l’angle de l’architecture d’un système multimodal dans son ensemble, puis sous l’angle de la modélisation de la fusion multimodale et, enfin, sous un angle plus algorithmique. Le premier angle, architectural, s’intéresse aux exigences nécessaires pour une intégration profitable de moteurs de fusion, en utilisant les critères d’utilisabilité et d’expressivité pour caractériser les qualités et les inconvénients de différentes architectures. Trois différentes architectures pour la création d’interfaces multimodales sont ensuite étudiées. Le framework HephaisTK, qui a servi de base d’étude expérimentale pour ce travail de thèse, est finalement décrit. Le deuxième angle, la modélisation du dialogue, présente huit directives pour la création de langages de modélisation du dialogue multimodal et propose le langage SMUIML comme exemple d’un langage suivant ces directives. Le troisième angle, algorithmique, étudie la fusion multimodale proprement dite par le biais de l’implémentation de deux algorithmes de fusion : d’abord, un algorithme « meaning frames », ensuite un algorithme basé sur les modèles de Markov cachés (HMM). Une évaluation des performances de ces deux algorithmes à l’aide d’un outil de mesure intégré est présentée en dernière analyse. Par l’étude à l, The field of multimodal interaction grew during the last decade, as a consequence of the advent of innovative input interfaces, as well as the development of research fields such as speech recognition. However, multimodal fusion and combination did not evolve at the same rate, which lead to a chasm between the use of input modalities and the different possibilities of combining them. This PhD thesis seeks to reduce the chasm between interaction means and fusion of their data. Fusion of multimodal input is approached in a global way in this research: first, from the point of view of the architecture of a multimodal system as a whole, then, from the point of view of multimodal dialog modeling, and finally from an algorithmic point of view. The architectural angle focuses on necessary features of an architecture to allow beneficial integration of a fusion engine, using usability and expressivity as attributes to characterize qualities and drawbacks of different architectures. Three different architectures targeting the creation of multimodal interfaces are subsequently studied. The HephaisTK framework, which served as the experimental foundation for this thesis work, is described in detail. The second angle, dialog modeling oriented, presents eight guidelines for creation of multimodal dialog modeling languages, then proposes the SMUIML language as an example of a language following those guidelines. Finally, the algorithmic angle studies multimodal fusion itself through the implementation of two fusion algorithms: an algorithm based on meaning frames and an algorithm based on hidden Markov models (HMMs). An evaluation of the performances of those two algorithms with the help of a proposed integrated benchmarking tool is also presented. Through theoretical as well as practical study of these three angles – architecture, modeling, algorithms – the different issues of multimodal fusion were defined and clarified. This newly elucidated knowledge combined with the software

Le domaine de l’interaction multimodale a pris beaucoup d’importance lors de cette dernière décennie, du fait de la création et de la mise sur le marché de nombreuses interfaces novatrices, ainsi que de l’évolution de domaines de recherche tels que la reconnaissance vocale. Cependant, la fusion et la combinaison de modalités n’ont pas évolué avec la même vigueur, aboutissant à un fossé entre l’utilisation de modalités d’entrée et les différentes possibilités de les combiner. Cette thèse de doctorat s’attache à réduire ce fossé entre les dispositifs d’interactions à disposition et la fusion de leurs données. Dans cette recherche, la fusion d’inputs multimodaux est approchée dans sa globalité : en premier lieu, sous l’angle de l’architecture d’un système multimodal dans son ensemble, puis sous l’angle de la modélisation de la fusion multimodale et, enfin, sous un angle plus algorithmique. Le premier angle, architectural, s’intéresse aux exigences nécessaires pour une intégration profitable de moteurs de fusion, en utilisant les critères d’utilisabilité et d’expressivité pour caractériser les qualités et les inconvénients de différentes architectures. Trois différentes architectures pour la création d’interfaces multimodales sont ensuite étudiées. Le framework HephaisTK, qui a servi de base d’étude expérimentale pour ce travail de thèse, est finalement décrit. Le deuxième angle, la modélisation du dialogue, présente huit directives pour la création de langages de modélisation du dialogue multimodal et propose le langage SMUIML comme exemple d’un langage suivant ces directives. Le troisième angle, algorithmique, étudie la fusion multimodale proprement dite par le biais de l’implémentation de deux algorithmes de fusion : d’abord, un algorithme « meaning frames », ensuite un algorithme basé sur les modèles de Markov cachés (HMM). Une évaluation des performances de ces deux algorithmes à l’aide d’un outil de mesure intégré est présentée en dernière analyse. Par l’étude à l, The field of multimodal interaction grew during the last decade, as a consequence of the advent of innovative input interfaces, as well as the development of research fields such as speech recognition. However, multimodal fusion and combination did not evolve at the same rate, which lead to a chasm between the use of input modalities and the different possibilities of combining them. This PhD thesis seeks to reduce the chasm between interaction means and fusion of their data. Fusion of multimodal input is approached in a global way in this research: first, from the point of view of the architecture of a multimodal system as a whole, then, from the point of view of multimodal dialog modeling, and finally from an algorithmic point of view. The architectural angle focuses on necessary features of an architecture to allow beneficial integration of a fusion engine, using usability and expressivity as attributes to characterize qualities and drawbacks of different architectures. Three different architectures targeting the creation of multimodal interfaces are subsequently studied. The HephaisTK framework, which served as the experimental foundation for this thesis work, is described in detail. The second angle, dialog modeling oriented, presents eight guidelines for creation of multimodal dialog modeling languages, then proposes the SMUIML language as an example of a language following those guidelines. Finally, the algorithmic angle studies multimodal fusion itself through the implementation of two fusion algorithms: an algorithm based on meaning frames and an algorithm based on hidden Markov models (HMMs). An evaluation of the performances of those two algorithms with the help of a proposed integrated benchmarking tool is also presented. Through theoretical as well as practical study of these three angles – architecture, modeling, algorithms – the different issues of multimodal fusion were defined and clarified. This newly elucidated knowledge combined with the software

This article introduces the problem of modeling multimodal interaction, in the form of markup languages. After an analysis of the current state of the art in multimodal interaction description languages, nine guidelines for languages dedicated at multimodal interaction description are introduced, as well as four different roles that such language should target: communication, configuration, teaching and modeling. The article further presents the SMUIML language, our proposed solution to improve the time synchronicity aspect while still fulfilling other guidelines. SMUIML is finally mapped to these guidelines as a way to evaluate their spectrum and to sketch future works.

This article introduces the problem of modeling multimodal interaction, in the form of markup languages. After an analysis of the current state of the art in multimodal interaction description languages, nine guidelines for languages dedicated at multimodal interaction description are introduced, as well as four different roles that such language should target: communication, configuration, teaching and modeling. The article further presents the SMUIML language, our proposed solution to improve the time synchronicity aspect while still fulfilling other guidelines. SMUIML is finally mapped to these guidelines as a way to evaluate their spectrum and to sketch future works