Search

International audience; We propose a hierarchical framework for collaborative intelligent systems. This framework organizes research challenges based on the nature of the collaborative activity and the information that must be shared, with each level building on capabilities provided by lower levels. We review research paradigms at each level, with a description of classical engineering-based approaches and modern alternatives based on machine learning, illustrated with a running example using a hypothetical personal service robot. We discuss cross-cutting issues that occur at all levels, focusing on the problem of communicating and sharing comprehension, the role of explanation and the social nature of collaboration. We conclude with a summary of research challenges and a discussion of the potential for economic and societal impact provided by technologies that enhance human abilities and empower people and society through collaboration with Intelligent Systems.; Nous proposons un cadre hiérarchique pour les systèmes intelligents collaboratifs. Ce cadre organise les défis de recherche en fonction de la nature de l'activité collaborative et des informations qui doivent être partagées, chaque niveau s'appuyant sur les capacités fournies par les niveaux inférieurs. Nous passons en revue les paradigmes de recherche à chaque niveau, avec une description des approches classiques basées sur l'ingénierie et des alternatives modernes basées sur l'apprentissage automatique, illustrées par un exemple courant utilisant un hypothétique robot de service personnel. Nous discutons des questions transversales qui se produisent à tous les niveaux, en nous concentrant sur le problème de la communication et du partage de la compréhension, le rôle de l'explication et la nature sociale de la collaboration. Nous concluons par un résumé des défis de la recherche et une discussion sur le potentiel d'impact économique et sociétal fourni par les technologies qui améliorent les capacités humaines et autonomisent les personnes et la société grâce à la collaboration avec les systèmes intelligents.

The increase in energy demand has motivated a large body of research, using a diverse collection of approaches to reduce domestic energy use. One promising approach to this societal problem is the development of systems that cooperate with the residents combined with digital behavior changes interventions. In this chapter, we present the Mondrian User Interface pattern intended to help system developers to structure and populate the interactive components of their system in order to inspire eco-responsibility. The Mondrian UI serves as an ambient decorative display that supports different levels of users’ attention investment while sustaining daily use. Like Mondrian’s Collections, the Mondrian UI pattern divides the screen into rectangles of different sizes painted in primary colors. The rectangles contain information that can be zoomed in and zoomed out to support multilevel representations of the same semantic information while maintaining informational context accessible. In order to minimize attentional effort, the top-level content of a rectangle is intended to communicate information at a glance while the other levels provide analytic details on demand with minimalist one-click interaction. Each rectangle of the Mondrian UI is populated with eco-inspiring content such as spatiotemporal information about energy use, recommendations and explanations, and social incentives, or with utilitarian content, such as the weather forecast, that family members have the habit of consulting frequently. Utilitarian content along with the display located in a socializing place is intended to enforce the daily use of eco-inspiring information. We have applied the Mondrian UI pattern to the design and implementation of the user interface of an e-coach electricity management system.

International audience; This work addresses the evaluation of interaction techniques for decision-making tasks performed by non-experts in the context of multi-objective optimization problems. Such tasks require making trade-offs between antagonistic criteria, according to individual subjective preferences. Evaluating such techniques is made difficult by the subjective nature of such tasks, as well as by a lack of rigorous methods for assessment. Our primary contributions to this problem are two-fold: (1) a set of subjective metrics including decision accuracy, choice satisfaction, and incentives to explore; (2) the use of a pragmatic approach to map these subjective metrics onto objective quantitative measures. To illustrate how this subjective-objective mapping can be performed in a pragmatic manner, we have conducted an experiment involving 177 participants to objectively measure and compare two multi-slider interaction techniques for decision-making tasks performed by non-experts in the context of domestic energy management. The results of this evaluation constitute a secondary contribution.

International audience; Reducing energy consumption is an individual and collective challenge that requires people to be proactive and fully involved. However, most approaches and solutions to this problem promote automated and autonomous systems that take full control of the decisions. Although these systems relieve users from setting the temperature in offices and homes, the level of thermal comfort chosen by the technology may not meet the individuals' requirements. Automating control is an obvious solution but it keeps the human out of the loop. In this work, an alternative approach is promoted based on system-human cooperation that allows people to take the final decision. However, to make sound decisions, users need to understand the system functioning and rationale and to be convinced and motivated to change their habits with regard to energy consumption. Our approach is based on an e-coach system, called Involved, that provides users with contextual explanations along with a user interface designed to persuade and keep the user involved. The results of this research are the development of an early prototype that provides end-users with a 24 hour-plan of recommended actions along with contextual explanations that justify each action. This plan satisfies the inhabitant’s preferred compromise between thermal comfort, air quality and financial cost specified by the inhabitant (i.e. user) using a novel interaction technique, the Trade-Off-Pareto sliders. Users can edit the plan (e.g. suppress an action), skip some actions, perform additional actions, or even change their preferred compromise, and be informed in real time of the consequences on energy consumption and comfort.

International audience; Most previous research in sustainable HCI has focused on electricity consumption in domestic environments or private office spaces. Here, we address use of lighting in public areas of office buildings with the goal of understanding and measuring how calm technologies can inspire positive engagement by promoting awareness and competition. We have conducted a 15-month study with the design, deployment, and assessment of two complementary ambient apparatus (one centralized, one distributed) in an office building occupied by ICT start-up companies. Our results show that calm technology can be effective under specific conditions, resulting in a significant reduction of average electricity consumption. We also discovered that in the absence of automatic controls, approximately 25% of lighting consumption occurred during off-work hours.

Housing consumption keeps growing in spite of energy efficient devices. The problem seems to come from not only the lack of information about how to use these devices but also from the lack of willingness from inhabitants. This double assessment invites to think about support toward inhabitants' behaviors concerning to energy consumption, which is affected by both devices and values. The paper follows an interdisciplinary research between persuasive human computer interaction and sociological studies of domestic energy consumption. The scientific contribution is the integration of the sociological dimensions of energy consumption and behavior change into interactive system design principles. We present a proof of concept for the sustainable change of users' behavior in energy consumption that demonstrates the applicability of these principles.

International audience; We present an experience with the development and evaluation of AppsGate, an ecosystem for the home that can be programmed by end-users. We show the benefits from using the homes of the project team members as real-life living-labs. In particular, we discuss the first person perspective experience as an effective way to conduct longitudinal experiments in real world settings. We conclude that a programmable habitat is desirable provided that attention cost is minimized.; Cet article présente une expérience avec la mise en œuvre et l’évaluation d’AppsGate, un écosystème domestique programmable par l’habitant. Nous montrons l’apport de l’utilisation des domiciles de membres du projet tout au long du processus de développement, et notamment l’intérêt de « vivre avec » comme technique d’expérimentation longitudinale.

International audience; We propose an approach based on Adaptive Multi-Agent Systems, using the principles of Meta-User Interfaces and Opportunism in order to solve Human-Computer Interaction Composition in Ambient interactive spaces. The idea of this approach is to see every component as an agent able to interact with other components to compose autonomously in order to opportunistically suggest to users smart compositions of his interactive ambient environment. We present the notions of component, composition, and human-computer interaction composition. We chose mainly two aspects of the composition of human-computer interaction which are the controllability and finality of the composition. Finally, we illustrate our approach with use cases taken from the neoCampus project.

International audience; Dans ce papier nous proposons une approche basée sur un système multi-agents adaptatif, en utilisant les principes de Méta-Interactions Homme-Machine et d'Opportunisme dans le but de résoudre le problème de la Composition d'Interactions Homme-Machine dans les espaces interactifs ambiants. L'idée de cette approche est de voir chaque composant comme un agent capable d'interagir avec les autres composants pour composer de manière autonome et ainsi suggérer à l'utilisateur de manière opportuniste des compositions de son environnement ambiant interactif. Nous avons choisi de présenter principalement deux aspects de la composition d'interactions homme-machine, la contrôlabilité et l'objectif de la composition. Enfin nous illustrons notre approche avec des cas d'utilisation dans le cadre d'un projet nommé neoCampus. Mots-clés informatique ubiquitaire ; espaces ambiants interactifs ; interactions homme-machine ; composition opportuniste de composants ; méta-interactions homme-machine ; théorie des systèmes multi-agents adaptatifs

The authors present their "lived-with" experience with an end-user development (EUD) prototype deployed in their home and show how the results overlap and complement findings from more traditional approaches to the study of EUD for the home. This article is part of a special issue on domestic pervasive computing.

Designing complex interactive systems requires the collaboration of actors with very different background. As a result, several languages and tools are used in a single project with no hope for interoperability. In this article, we examine whether a universal language is a realistic approach to UI specification by looking for answers into the domain of Linguistics while finding analogies in software engineering. Then, we explore one particular avenue from main-stream software engineering: that of Model Driven Engineering where the notion of transformation is key to the definition of bridges between languages and tools. Building upon these two analyses, we then show how model-driven engineering can be successfully exploited in the development and execution of plastic multimodal UIs illustrated with a variety of complementary tools.

International audience; In this paper we propose an ecological view in which a smart home is seen as an interconnected collection of smart objects that work together to provide services to inhabitants. We review home technologies in a historical context in which the home is a personal habitat that provides services to inhabitants , and draw lessons from the profusion of new services that were made possible by the introduction of electricity in the home during the 20th century. We examine possible metaphors for smart homes, including the smart home as an inside-out autonomous robot, and the smart home as an ecosystem of smart objects providing services. We propose a taxonomy for smart home services and discuss examples for each class of service. We conclude with a discussion of required system qualities and potential show-stoppers. Continued advances in information and communication technologies, coupled with progress in machine learning, sensors, actuators and human computer interaction make it increasingly easy to embed technologies for perception, action, communication and interaction in ordinary human objects. The result is an enabling technology for smart objects and smart environments with the potential to provide revolutionary new services. In this paper we discuss how this technology can be used to create new forms of intelligent services for the home. We begin by discussing historical barriers to Home Automation, and propose an alternative ecological view of the home as a personal habitat that provides services such as personal protection and shelter from the elements. We examine the profound rupture in the nature of services that resulted from the introduction of electricity in the home at the beginning of the 20 th century and draw lessons from the adoption of different forms of electric appliances. We then examine the nature of services that are made possible by the introduction of ambient intelligence in the home. We propose a taxonomy for smart home services in terms of tools, housekeepers, advisors, and media. For each class, we explore forms of services for different functional areas of the home. We conclude by discussing required system qualities

Context is not simply the state of a predefined environment with a fixed set of interaction resources. It's part of a process of interacting with an ever-changing environment composed of reconfigurable, migratory, distributed, and multiscale resources.

Mobility coupled with the development of a wide variety of access devices has engendered new requirements for HCI such as the ability of user interfaces (UI) to adapt to different contexts of use. By context of use, we mean the triple . This paper first analyses the problem space of adaptation then proposes a reference framework covering both the development and execution of plastic UI. This framework helps in structuring the state of the art. It emphasises two open issues that we propose to address by an agent and conceptual graphs-based approach. The paper is illustrated on a running example CamNote that supports the presentation of slides in different contexts of use.

This paper describes a framework that serves as a reference for classifying user interfaces supporting multiple targets, or multiple contexts of use in the field of context-aware computing. In this framework, a context of use is decomposed in three facets: the end users of the interactive system, the hardware and software computing platform with which the user have to carry out their interactive tasks and the physical environment where they are working. Therefore, a context-sensitive user interface is a user interface that exhibits some capability to be aware of the context (context awareness) and to react to changes of this context. This paper attempts to provide a unified understanding of context-sensitive user interfaces rather than a prescription of various ways or methods of tackling different steps of development. Rather, the framework structures the development life cycle into four levels of abstraction: task and concepts, abstract user interface, concrete user interface and final user interface. These levels are structured with a relationship of reification going from an abstract level to a concrete one and a relationship of abstraction going from a concrete level to an abstract one. Most methods and tools can be more clearly understood and compared relative to each other against the levels of this framework. In addition, the framework expresses when, where and how a change of context is considered and supported in the context-sensitive user interface thanks to a relationship of translation. In the field of multi-target user interfaces is also introduced, defined, and exemplified the notion of plastic user interfaces. These user interfaces support some adaptation to changes of the context of use while preserving a predefined set of usability properties

This paper presents early lessons from the development of SPOK, an End-User Development Environment for smart homes. SPOK (Simple PrOgramming Kit) uses a pseudo-natural language as an end-user programming language and runs on top of an extension of OSGi/iPOJO to support the dynamic and resilient management of web services and devices from a variety of protocols including EnOcean, UPnP, and Watteco. The motivation for SPOK is to give the power back to end-users so that they can shape their own smart home at will. This paper reports lessons learned from the methods we have used to validate our hypotheses as well as a number of technical issues concerning development of this type of EUDE. A Video of SPOK in action as of October 2013 is accessible at: http://iihm.imag.fr/demos/appsgate/appsgate2013.mp4

Software tools for tile construction of interactive systems such as user interface toolkits, application skeletons, and user interface generators, alleviate the activity of programming but do not eliminate software architecture modelling. For example, the "callback" mechanism made popular by X-Window, does not enforce the distinction between domain-specific concepts and presentation-specific issues. Without a framework for prescribing the appropriate usage of callbacks, the software designer may develop an interactive system that cannot be maintained nor modified. Cooperative systems, which draw upon a wide variety of tools, techniques, and constraints, make architecture modelling even more crucial.In order to reason about software modelling for cooperative systems, we first need to clarify the nature of the problem space. The role of a problem space is to identify the set of concepts relevant to a particular domain of interest and to organize these concepts into a framework suitable for rising the appropriate questions, for clarifying the scope of the solution space, or for characterizing a particular design solution. The litterature reveals a wide variety of issues related to CSCW but these concepts are not organized in a way that supports software architecture modelling. We claim that one goal of the workshop is to define a problem space useful for reasoning about software architecture and use this framework as foundations for comparing, devising, and assessing current and future architecture models.In section 2, we present a draft proposal of the problem space for the software design of cooperative systems. In section 3, we describe PAC-Amodeus, our own conceptual architecture model, and show how it can support the issues made explicit in our problem space.

National audience; We present KISS (Knit your Ideas Into Smart Spaces), an end-user development system for the home. KISS enables users to program their home with sentences expressed in a pseudo-natural language. Programs can be tested either with the virtual home or in the real home. In this paper, we describe KISS and report an evaluation that shows that users are able to program a real-life scenario.; Nous présentons KISS (Knit your Ideas Into Smart Spaces), un outil de développement d'habitat intelligent par l'utilisateur final. KISS permet de programmer son habitat en termes de phrases exprimées en langage pseudonaturel. La mise au point de ces programmes peut se faire in situ dans l'appartement réel ou dans sa réplique virtuelle. L'évaluation de KISS montre que les utilisateurs parviennent à programmer un scénario réaliste.

International audience; Weiser's vision has been used to justify and define new research within an extremely diverse collection of fields. It is increasingly evident that such research cannot be carried out in isolation. Research in this area is fundamentally multi-disciplinary, requiring the assimilation of problems and concepts from a variety of specializations. From our perspective, in its current state, ambient intelligence is only the latest stage in the evolution of informatics as a scientific discipline. In this chapter, we will provide an overview of the field in its current state.

Session: Multimodal interfaces; International audience; A large variety of gestural interaction techniques based on accelerometers is now available. In this article, we propose a new taxonomic space as a systematic structure for supporting the comparative analysis of these techniques as well as for designing new ones. An interaction technique is plotted as a point in a space where the vertical axis denotes the semantic coverage of the techniques, and the horizontal axis expresses the physical actions users are engaged in, i.e. the lexicon. In addition, syntactic modifiers are used to express the interpretation process of input tokens into semantics, as well as pragmatic modifiers to make explicit the level of indirection between users' actions and system responses. To demonstrate the coverage of the taxonomy, we have classified 25 interaction techniques based on accelerometers. The analysis of the design space per se reveals directions for future research.

24 pages; International audience; The increasing use of the Web as a software platform together with the advance of technology has promoted Web applications as a starting point for improving communication between citizens and administration. Currently, several e-government Web portals propose applications for accessing information regarding healthcare, taxation, registration, housing, agriculture, education, and social services, which otherwise may be difficult to obtain. However, the adoption of services provided to citizens depends upon how such applications comply with the users' needs. Unfortunately, building an e-government website doesn't guarantee that all citizens who come to use it can access its contents. These services need to be accessible to all citizens/customers equally to ensure wider reach and subsequent adoption of the e-government services. User disabilities, computer or language illiteracy (e.g., foreign language), flexibility on information access (e.g., user remotely located in rural areas, homeless, mobile users), and ensuring user privacy on sensitive data are some of the barriers that must be taken into account when designing the User Interface (UI) of e-government applications. Although several initiatives (such as the W3C WAI) focus on how to promote usability and accessibility of content provided via e-government, many governments are enhancing their technology to make their services compatible with new communication channels available through multiple devices including interactive digital TVs (iTV), personal digital assistants (PDAs), and mobile phones. In this chapter we focus on this latter issue, which means the development of multitarget government services available across several platforms. Hereafter we discuss the major constraints underlining the importance of investment on the UI's design of e-government applications. Moreover, we propose a framework for envisioning advanced UIs where the adaptation to the user's capabilities and available devices as well as physical and social environment will play a major role.

This paper reports on an empirical study performed to validate a theoretical model about the composition of smart artifacts by end-users. We have solicited 17 families and used a combination of interviews and a playful cultural probe. Results show that families are willing to couple smart objects to improve their lives, and that the theoretical questions raised by our model are sound. New services are more easily envisioned when coupling involves a "communicating" object or a "programmable" object. Being a communicating object has more impact than being programmable. Services envisioned by families included: Service substitution (e.g., replacing the user interface of the alarm clock with that of the time setting available on mobile phones); Service improvement (e.g., extending appliances such as the washing machine, by coupling them with the interaction resources of the TV set); Service chaining as ready for use "packages" to improve well-being for routine activities; along with Service "starters" as a means to control packages. Other findings are related to the playful cultural probe. Based on pictures of objects taken by the family members, our probe served as an ice-breaker; family members "revealed their house" naturally and the use of images of intimate objects increased interest and imagination.

International audience; How can people identify the services that they might expect from their smart home when they have little to no knowledge about novel technologies? This paper reports on a user needs analysis method designed to answer this question: DisQo. We have recruited 17 families and used a combination of interviews and playful cultural probes. Results show that families are willing to couple smart objects to improve their lives.

National audience; Ce chapitre structure les éléments clés de l'évolution des IHM en intelligence ambiante. Il montre, selon une perspective utilisateur, que la plasticité se pose finalement comme intégrateur de ces évolutions. La plasticité est ensuite illustrée dans le domaine des transports qui se révèle être un terrain extrêmement fertile à son application.

Ubiquitous computing introduces new constraints on the way we build interactive systems. Traditionally centralized, user interfaces (UI) can be distributed across a dynamic set of heterogeneous devices. In this article, we examine these new kinds of UIs, named plastic UIs, from a software engineering perspective. They are dynamic adaptable distributed software, and they able to deal with device heterogeneity at multiple levels of abstraction. However, the generic solutions from distributed software do not take into account the specificity of human-computer interaction. To address these limitations, we propose Ethylene, a conceptual and technical framework to develop UIs for ubiquitous computing. Our solution is based on the integration of model driven engineering with a particular combination of component and service oriented approaches, as weel as an original way to overcome the heterogeneity of communication models between components.

In this article, we propose a model for networked artifacts inspired from molecular chemistry. It demonstrates the intrinsic complexity of the domain illustrated with unsolved problems such as mastering the semantics of networked artefacts. Based on this model, we identify similarities with the service-oriented computing paradigm and suggest possible avenues for collaboration between Software Engineering and researchers in EUD including: semantic alignment between end-user programming languages and service descriptions, and human-service interaction as a transversal issue in software design.

Distributed user interfaces (DUIs) are those interfaces whose different parts can be distributed in time and space on different monitors, screens, and computing platforms, depending on several parameters expressing the context of use, such as the user, the computing platform, and the physical environment in which the user is carrying out her interactive task To understand and classify existing approaches for DUIs and to identify under explored situations of DUIs, a reference model for DUIs is introduced that examines DUIs according to four 'C' dimensions: computation (what is distributed?), communication (when is it distributed?), coordination (who is it distributed?), and configuration (from where and to where is the distribution operated?). At the core of this reference model exists the original notion of user interface habitat, that is the place where a particular type of user interface is normally found. According to this notion, it is possible to explore and investigate a wide spectrum of DUIs among which we exemplify several cases coming from our existing research and development of DUIs.

In this chapter, we bring together the main results from the developments in plastic user interfaces (UI) and in multimodal UIs using a theoretic and conceptual perspective as a unifying approach. In particular, we provide an operational model for the notion of modality as well as for the combination of multiple modalities in terms of the CARE properties. Then, we clarify the problem space of UI plasticity and show how multimodality can be used as a means for UI adaptation. To address the challenges raised by the development of plastic multimodal UIs, we propose three principles that have been applied to several proof-of-concept exemplars: (1) Blur the distinction between design-time and run-time, (2) Mix close and open adaptiveness and (3) Keep humans in the loop.

Plastic User Interfaces (UI) are able to adapt to their context of use while preserving usability. Research efforts have focused so far, on the functional aspect of UI adaptation, while neglecting the usability dimension. This paper investigates how the notion of mapping as promoted by Model Driven Engineering (MDE), can be exploited to control UI adaptation according to explicit usability criteria. In our approach, a run-time UI is a graph of models related by mappings. Each model (e.g., the task model, the Abstract UI, the Concrete UI, and the final UI) describes the UI from a specific perspective from high-level design decisions (conveyed by the task model) to low-level executable code (i.e. the final UI). A mapping between source and target models specifies the usability properties that are preserved when transforming source models into target models. This article presents a meta-model for the notion of mapping and shows how it is applied to plastic UIs.

Coupling is the action of binding two entities so that they can operate together to provide new functions. In this article, we propose a formal definition for coupling and present two complementary conceptual tools to reason about coupling interaction resources. The first tool is a graph theoretic and algebraic notation that can be used to identify the consequents of causal couplings so that the side-effects of the creation of a coupling can be analyzed in a formal and systematic way. The second tool formulates the problem of coupling using an 8 state automaton that models the life cycle of a coupling and provides designers with a structure to verify that usability properties have been satisfied for each state. We conclude with the concept of meta-UI, an overarching interactive system that shows that coupling is only one aspect of a larger problem space.

In this paper, we describe the COMETs Inspector, a software tool providing user interface designers and developers with a semantic network in order to control the plasticity of their User Interfaces (UI) at run-time. Thanks to a set of predefined relationships, the semantic network links together various concepts ranging from the final UI (i.e., the UI described in terms of technological spaces) to the concrete and abstract UIs (i.e., the UI respectively described in terms of concrete interaction objects independently of any technological space, and abstract individual components and containers independently of any interaction modality) up to the tasks and concepts of the interactive system. In this way, plasticity can be addressed at four levels of abstraction (task and concepts, abstract, concrete, and final user interface) for forward, reverse, and lateral engineering. The end user exploits the semantic network at run time to adapt his/her UI to another context of use by identifying, selecting, and applying plasticity suitable operations.

In this article, we propose the concept of meta-User Interface (meta-UI) as the set of functions (along with their user interfaces) that are necessary and sufficient to control and evaluate the state of interactive ambient spaces. This set is meta-, since it serves as an umbrella beyond the domain-dependent services that support human activities in an ambient interactive space. They are User Interface-oriented since their role is to help users to control and evaluate the state of this space. We present a dimension space to classify, compare, and contrast disparate research efforts in the area of meta-UI's. We then exploit the generative power of our design space to suggest directions for future research.

User Interface (UI) plasticity denotes UI adaptation to the context of use (user, platform, physical and social environments) while preserving usability. In this article, we focus on the use of Model-Driven Engineering and demonstrate how the intrinsic flexibility of this approach can be exploited by designers for UI prototyping as well as by end-users in real settings. For doing so, the models developed at design-time, which convey high-level design decisions, are still available at run-time. As a result, an interactive system is not limited to a set of linked pieces of code, but is a graph of models that evolves, expresses and maintains multiple perspectives on the system from top-level tasks to the final UI. A simplified version of a Home Heating Control System is used to illustrate our approach and technical implementation.

User interface (UI) plasticity denotes UI adaptation to the context of use (user, platform, physical and social environment) while preserving usability. Our approach to this problem is to bring together MDE (Model Driven Engineering) and SOA (Service Oriented Approach) within a unified framework that covers both the development stage and the runtime phase of plastic UI's. In particular, an interactive system is modelled as a graph of models that can be dynamically manipulated by, and/or encapsulated as services.

Coupling is the action of binding two entities so that they can operate together to provide new functions. In this article, we propose a formal definition for coupling and present a graph theoretic notation so that the side-effects of the creation of a coupling can be analyzed in a formal and systematic way. We then describe I-AM (Interaction Abstract Machine), a middleware that supports the dynamic coupling of interaction resources such as screens, keyboards and mice, to form a unified interactive space. Using our notation, we illustrate how couplings are supported in I-AM.

Three types of representation are typically produced during the User Interface (UI) development life cycle: a conceptual representation holding the models used for elaborating a UI, an internal representation concerning the code of the UI, and an external representation expressing the look and feel of the UI. While the end user typically manipulates the external representation only, the designer and the developer respectively focus on the conceptual and internal representations. The Comets Inspector gathers all three representations into a single environment, thus providing the user (actually, the designer and the developer; in the future, the end-user) with multiple views of a same UI simultaneously. Each representation is made observable and modifiable through one or many “mini-UIs”. Consistency is ensured through mappings between these representations. From a methodological point of view, the benefit is the integration of three stakeholders’ perspectives in a consensual and consistent way, enabling the exploration and manipulation of design alternatives at run time. In particular, when the context of use will be changing, the end-user will be able to inspect the UI capabilities and control its adaptation, thus sustaining explicit plasticity

This paper presents SIROCO, a research on user interface design and development support in an object-oriented programming environment. User interface (UI) design in SIROCO focuses on conceptual modeling; a fully object-oriented conceptual representation model and language is proposed, distinguishing the function dimension from the use dimension in an interactive system. Interaction style and presentation aspects are dealt with as generic parameters. SIROCO promotes a continuous object-oriented approach from UI design to UI implementation. Transition from conceptual design to implementation is achieved through an extended object-oriented software architecture model. Effective UI development is supported by an object factory that produces the object code needed to implement the specified user interface.

Ubiquitous computing has introduced the need for interactive systems to run on multiple platforms in different physical environments. Providing a user interface specially crafted for each context of use ( ) is costly, may result in inconsistent behaviors [5] and above all is limited to the contexts of use that have been envisioned at design time. Opportunistic interaction in a changing environment requires enabling approaches and techniques for gracefully adapting the interactive system to its context of use. In Human-Computer Interaction, this ability of an interactive system to withstand variations of context of use while preserving its quality in use is called plasticity [7]. This paper shows how Model Driven Engineering is suited for supporting plasticity both at design time and run time.

This paper addresses the problem of coupling interaction resources. Coupling is the action of binding two entities so that they can operate together to provide a new set of functions. For example, coupling a piece of cardboard with a steerable camera-projector pair results in a portable interactive surface. Although coupling is not a new phenomenon, recent research demonstrates that coupling opens the way to unbounded forms of interaction. Because the risk of introducing complexity is high, we propose an analytical model that can serve two purposes: the model can be used to inform the design of novel user interfaces; the computational version of the model can be used at run time as a mechanism to evaluate candidate solutions when dynamic adaptation to the context of use must be undertaken. We illustrate the contribution of the model with three chosen running examples.

This chapter addresses the plasticity of user interfaces from the software development perspective. We propose a conceptual framework that helps structure the development process of plastic user interfaces. This framework helps to clearly characterize the functional coverage of current tools such as ARTStudio and our own development environment for plastic user interfaces; it also helps to identify requirements for future tools.

International audience; This paper describes the “FAME” multi-modal demonstrator, which integrates multiple communication modes – vision, speech and object manipulation – by combining the physical and virtual worlds to provide support for multi-cultural or multi-lingual communication and problem solving. The major challenges are automatic perception of human actions and understanding of dialogs between people from different cultural or linguistic backgrounds. The system acts as an information butler, which demonstrates context awareness using computer vision, speech and dialog modeling. The integrated computerenhanced human-to-human communication has been publicly demonstrated at the FORUM2004 in Barcelona and at IST2004 in The Hague. Specifically, the “Interactive Space” described features an “Augmented Table” for multi-cultural interaction, which allows several users at the same time to perform multi-modal, cross-lingual document retrieval of audio-visual documents previously recorded by an “Intelligent Cameraman” during a week-long seminar.

This paper addresses software adaptation to context of use. It goes one step further than our early work on plasticity [5]. Here, we propose a revision of the notion of software plasticity that we apply at the widget level in terms of comets. Plasticity is defined as the ability of an interactive system to withstand variations of context of use while preserving quality in use where quality in use refers to the ISO definition. Plasticity is not limited to the UI components of an interactive system, nor to a single platform: adaptation to context of use may also impact the functional core, it may have an effect on the nature of the connectors, and it may draw upon the existence of multiple platforms in the vicinity to migrate all or portions of the interactive system. A new reference framework that structures the development process of plastic interactive systems is presented to cover these issues. The framework is then applied at the granularity of widgets to provide the notion of a comet. A comet is an introspective widget that is able to self-adapt to some context of use, or that can be adapted by a tier-component to the context of use, or that can be dynamically discarded (versus recruited) when it is unable (versus able) to cover the current context of use. To do so, a comet publishes the quality in use it guarantees, the user tasks and the domain concepts that it is able to support, as well as the extent to which it supports adaptation.

This paper defines the problem space of distributed, migratable and plastic user interfaces, and presents CAMELEON-RT, a technical answer to the problem. CAMELEON-RT1 is an architecture reference model that can be used for comparing and reasoning about existing tools as well as for developing future run time infrastructures for distributed, migratable, and plastic user interfaces. We have developed an early implementation of a run time infrastructure based on the precepts of CAMELEON-RT.

Without the support of adequate software infrastructures, the implementation of context sensitive interactive systems is hard to achieve in a sound way. In this article, we propose the notion of contextor, a software abstraction that supports the operational deployment of interaction contexts. We show how contextors are organized into levels of abstraction, and how these levels fit within the Arch architecture reference model for interactive systems. Based on the P2P paradigm, the contextor infrastructure is intended to support both mobility and ubiquity. Similar in spirit to the Context Toolkit, we make the differences explicit.

Although coupling interaction resources is key to ubiquitous computing, this notion has been overlooked in previous studies. In this paper, we address this notion in a more systematic way by describing coupling as a finite state machine whose states should be maintained observable and controllable by users in a flexible and reliable manner. We illustrate the discussion with state of the art demonstrators as well as with I-AM, a middleware that supports the dynamic construction of multi-screen, multi-keyboard, and multi-pointer interactive spaces.

Adaptation of User Interfaces (UI) to context of use is becoming a major issue in HCI. By context of use, we mean the triplet. Plasticity refers to the ability of UIs to withstand variations of context of use while preserving usability. Based on a reference framework that provides designers with a powerful tool for reasoning about plasticity, this paper addresses plasticity at the level of interactors. Comets (COntext sensitive Multi-target widgETS) are a new kind of widget that support adaptation at any level of abstraction: concepts and tasks, abstract, concrete and final UIs. This paper proposes a model as well as run-time software mechanisms that support the design of comets as well as run-time adaptation. Run-time is ruled according to a set of strategies and politics.

This article presents IntrosPAC, a software tool based on aspect oriented programming, which, from a PAC-Amodeus compliant source code, generates a graphic representation of the corresponding conceptual architecture. The graphic animation supports developers for understanding the structure of their code as well as for detecting violation of the principles of the reference architecture model. IntrosPAC has been successful in teaching software architecture modeling for interactive systems and in improving students understanding.