Search

We present Rataplan, a robust and resilient pixel-based approach for linking multi-modal proxies to automated sequences of actions in graphical user interfaces (GUIs). With Rataplan, users demonstrate a sequence of actions and answer human-readable follow-up questions to clarify their desire for automation. After demonstrating a sequence, the user can link a proxy input control to the action which can then be used as a shortcut for automating a sequence. Alternatively, output proxies use a notification model in which content is pushed when it becomes available. As an example use case, Rataplan uses keyboard shortcuts and tangible user interfaces (TUIs) as input proxies, and TUIs as output proxies. Instead of relying on available APIs, Rataplan automates GUIs using pixel-based reverse engineering. This ensures our approach can be used with all applications that offer a GUI, including web applications. We implemented a set of important strategies to support robust automation of modern interfaces that have a flat and minimal style, have frequent data and state changes, and have dynamic viewports.

When using computer-aided translation systems in a typical, professional translation workflow, there are several stages at which there is room for improvement. The SCATE (Smart Computer-Aided Translation Environment) project investigated several of these aspects, both from a human-computer interaction point of view, as well as from a purely technological side. This paper describes the SCATE research with respect to improved fuzzy matching, parallel treebanks, the integration of translation memories with machine translation, quality estimation, terminology extraction from comparable texts, the use of speech recognition in the translation process, and human computer interaction and interface design for the professional translation environment. For each of these topics, we describe the experiments we performed and the conclusions drawn, providing an overview of the highlights of the entire SCATE project. The research in this project was funded by the Flemish Agency for Innovation and Technology IWT, project number 13007.

Implementing multimodal interactions with event-driven languages results in a ‘callback soup', a source code littered with a multitude of flags that have to be maintained in a self-consistent manner and across different event handlers. Prototyping multimodal interactions adds to the complexity and error sensitivity, since the program code has to be refined iteratively as developers explore different possibilities and solutions. The authors present a declarative language for rapid prototyping multimodal interactions: Hasselt permits declaring composite events, sets of events that are logically related because of the interaction they support, that can be easily bound to dedicated event handlers for separate interactions. The authors' approach allows the description of multimodal interactions at a higher level of abstraction than event languages, which saves developers from dealing with the typical ‘callback soup' thereby resulting in a gain in programming efficiency and a reduction in errors when writing event handling code. They compared Hasselt with using a traditional programming language with strong support for events in a study with 12 participants each having a solid background in software development. When performing equivalent modifications to a multimodal interaction, the use of Hasselt leads to higher completion rates, lower completion times, and less code testing than when using a mainstream event-driven language.

Empathic communication allows individuals to perceive and understand the feeling and emotion of the person with whom they are interacting. This could be particularly important during remote collaboration (such as remote assistance or distance learning) to enhance the social and emotional understanding of geographically distributed partners. However, supporting awareness in remote collaboration is very challenging especially when the interaction with the remote parties results in less information that can be communicated than in a physical interaction. We explore the effect of visualization using physiological cues that allow users to interpret emotional behaviors of remote parties with whom they are interacting in real time. The proposed visual representation allows users to infer emotional patterns from physiological cues that can potentially influence their communication approach toward a more aggressive style or maintain passive and peaceful interaction. We conducted a study involving participants who were paired up for a collaborative assessment task, interacting via voice only, videoconference, or a visual representation of the physiological measurements. Participants perceived the usage of our visual representation with higher group cohesiveness than using voice-only interaction. Further analysis shows that the visual representation significantly increases the positive affect score (i.e., participants are perceived to be more alert and demonstrate less distress) during remote collaboration. We discuss the possibilities of the proposed visual representation to support empathic communication during remote collaboration, and the benefits to the remote partners of having positive affect and group cohesiveness.

Smart objects can be smart because of the information and communication technology that is added to human-made artifacts. It is not, however, the technology itself that makes them smart but rather the way in which the technology is integrated, and their smartness surfaces through how people are able to interact with these objects. Hence, the key challenge for making smart objects successful is to design usable and useful interactions with them. We list five features that can contribute to the smartness of an object, and we discuss how smart objects can help resolve the simplicity-featurism paradox. We conclude by introducing the three articles in this special issue, which dive into various aspects of smart object interaction: augmenting objects with projection, service-oriented interaction with smart objects via a mobile portal, and an analysis of input-output relations in interaction with tangible smart objects.

Localisation is a standard feature in many mobile applications today, and there are numerous techniques for determining a user’s location both indoors and outdoors. The provided location information is often organised in a format tailored to a particular localisation system’s needs and restrictions, making the use of several systems in one application cumbersome. The presented approach models the details of localisation systems and uses this model to create a unified view on localisation in which special attention is paid to uncertainty coming from different localisation conditions and to its presentation to the user. The work discusses technical considerations, challenges and issues of the approach, and reports on a user study on the acceptance of a mobile application’s behaviour reflecting the approach. The results of the study show the suitability of the approach and reveal users’ preference toward automatic and informed changes they experienced while using the application.

Professional video searchers typically have to search for particular video fragments in a vast video archive that contains many hours of video data. Without having the right video archive exploration tools, this is a difficult and time consuming task that induces hours of video skimming. We propose the video archive explorer, a video exploration tool that provides visual representations of automatically detected concepts to facilitate individual and collaborative video search tasks. This video archive explorer is developed by employing a user-centred methodology, which ensures that the tool is more likely to fit to the end user needs. A qualitative evaluation with professional video searchers shows that the combination of automatic video indexing, interactive visualisations and user-centred design can result in an increased usability, user satisfaction and productivity. Haesen M., Meskens J., Luyten K., Coninx K., Becker J.H., Tuytelaars T., Poulisse G.-J., Pham The P., Moens M.-F., ''Finding a needle in a haystack: an interactive video archive explorer for professional video searchers'', Multimedia tools and applications, vol. 63, no. 2, pp. 331-356, 2013. ispartof: Multimedia Tools and Applications vol:63 issue:2 pages:331-356 status: published

Design, and in particular user-centered design processes for interactive systems, typically involve multidisciplinary teams. The different and complementary perspectives of the team members enrich the design ideas and decisions, and the involvement of all team members is needed to achieve a user interface for a system that carefully considers all aspects, ranging from user needs to technical requirements. The difficulty is getting all team members involved in the early stages of design and communicating design ideas and decisions in a way that all team members can understand them and use them in an appropriate way in later stages of the process. This chapter describes the COMuICSer storyboarding technique, which presents the scenario of use of a future system in a way that is understandable for each team member, regardless of their background. Based on an observational study in which multidisciplinary teams collaboratively created storyboards during a co-located session, we present recommendations for the facilitation of co-located collaborative storyboarding sessions for multidisciplinary teams and digital tool support for this type of group work.

The increasing number of smart objects in our everyday life shapes how we interact beyond the desktop. In this workshop we discuss how advanced interactions with smart objects in the context of the Internet-of-Thingsshould be designed from various perspectives, such as HCI and AI as well as industry and academia.

Implementing multimodal interactive systems with event-driven tools requires splitting the code across multiple event handlers, which greatly complicate programmers' work. To alleviate this complexity we propose to extend the capabilities of existing event-driven tools so that instead of detecting user events as if they were independent, they are able to detect sequences of semantically related events defined by programmers. Such extended capability can only be possible with notations for (1) defining event patterns, and for (2) binding these patterns with one or multiple event handlers, and (3) specifying human-machine dialogs. This paper presents Hasselt UIMS, a tool that supports the creation of multimodal interfaces with these notations.

There is a growing demand for design support to create interactive systems that are deployed in ambient intelligent environments. Unlike traditional interactive systems, the wide diversity of situations these type of user interfaces need to work in require tool support that is close to the environment of the end-user on the one hand and provide a smooth integration with the application logic on the other hand. This paper shows how the model-based user interface development methodology can be applied for ambient intelligent environments; we propose a task-centered approach towards the design of interactive systems by means of appropriate visualizations and simulations of different models. Besides the use of these typical user interface models such as the task- and presentation-model to support interface design, we focus on user interfaces supporting situated task distributions and a visualization of context influences on deployed, possibly distributed, user interfaces. To enable this we introduce an environment model describing the device configuration at particular moment in time. To support the user interface designer while creating these complex interfaces for ambient intelligent environments, we discuss tool support using a visualization of the environment together with simulations of the user interface configurations. We also show how the concepts presented in the paper can be integrated within model-driven engineering, hereby narrowing the gap between HCI design and software engineering.

The usage of computing systems has evolved dramatically over the last years. Starting from a low level procedural usage, in which a process for executing one or several tasks is carried out, computers now tend to be used in a problem oriented way. Future computer usage will be more centered around particular services, and will not be focused on platforms or applications. These services should be independent of the technology used to interact with them. In this paper an approach will be presented which provides a uniform interface to such services, without any dependence on modality, platform or programming language. Through the usage of general user interface descriptions, presented in XML, and converted using XSLT, a uniform framework is presented for runtime migration of user interfaces. As a consequence, future services will become easily extensible for all kinds of devices and modalities. Special attention goes out to a component-based software development approach. Services represented by and grouped in components can offer a special interface for modal- and device-independent rendering. Components become responsible for describing their own possibilities and constraints for interacting. An implementation serving as a proof of concept, a runtime conversion of a joystick in a 3D virtual environment into a 2D dialog-based user interface, is developed.

The present paper describes an empirical user study intended to compare the programming efficiency of our proposed domain-specific language versus a mainstream event language when it comes to modify multimodal interactions. By concerted use of observations, interviews, and standardized questionnaires, we managed to measure the completion rates, completion time, code testing effort, and perceived difficulty of the programming tasks along with the perceived usability and perceived learnability of the tool supporting our proposed language. Based on this experience, we propose some guidelines for designing comparative user studies of programming languages. The paper also discusses the considerations we took into account when designing a multimodal interaction description language that intends to be well regarded by its users.

Omni-directional video (ODV) is a novel medium that offers viewers a 360o panoramic recording. This type of content will become more common within our living rooms in the near future, seeing that immersive displaying technologies such as 3D television are on the rise. However, little attention has been given to how to interact with ODV content. We present a gesture elicitation study in which we asked users to perform mid-air gestures that they consider to be appropriate for ODV interaction, both for individual as well as collocated settings. We are interested in the gesture variations and adaptations that come forth from individual and collocated usage. To this end, we gathered quantitative and qualitative data by means of observations, motion capture, questionnaires and interviews. This data resulted in a user-defined gesture set for ODV, alongside an in-depth analysis of the variation in gestures we observed during the study.

During remote video-mediated assistance, instructors often guide workers through problems and instruct them to perform unfamiliar or complex operations. However, the workers’ performance might deteriorate due to stress. We argue that informing biofeedback to the instructor, can improve communication and lead to lower stress. This paper presents a thorough investigation on mental workload and stress perceived by twenty participants, paired up in an instructor-worker scenario, performing remote video-mediated tasks. The interface conditions differ in task, facial and biofeedback communication. Two self-report measures are used to assess mental workload and stress. Results show that pairs reported lower mental workload and stress when instructors are using the biofeedback as compared to using interfaces with facial view. Significant correlations were found on task performance with reducing stress (i.e. increased task engagement and decreased worry) for instructors and declining mental workload (i.e. increased performance) for workers. Our findings provide insights to advance video-mediated interfaces for remote collaborative work.

We present a new interaction space for wearables by integrating interactive elements, in the form of buttons, into outdoor clothing, specifically jackets and coats. Interactive buttons, or 'iButtons', allow users to perform specific tasks using subtle, inconspicuous gestures. They are intended for outdoor settings, where reaching for a mobile phone or an other device may not be convenient or appropriate. Different types of buttons serve dedicated functions, and appropriate placement of these buttons make them easily accessible, without requiring visual contact. By adding context sensitivity, these buttons can also be repurposed to fit other functions. By linking multiple buttons, it is possible to create workflows for specific tasks. We provide a description of an initial iButton design space and highlight some scenarios to illustrate the envisioned usage of interactive buttons.

There are currently toolkits that allow the specification of executable multimodal human-machine interaction models. Some provide domain-specific visual languages with which a broad range of interactions can be modeled but at the expense of bulky diagrams. Others instead, interpret concise specifications written in existing textual languages even though their non-specialized notations prevent the productivity improvement achievable through domain-specific ones. We propose a domain-specific textual language and its supporting toolkit; they both overcome the shortcomings of the existing approaches while retaining their strengths. The language provides notations and constructs specially tailored to compactly declare the event patterns raised during the execution of multimodal commands. The toolkit detects the occurrence of these patterns and invokes the functionality of a back-end system in response.

Learning to play a musical instrument such as the piano requires a substantial amount of practice and perseverance in learning to read and play from sheet music. Our interactivity demo allows people to learn to play songs without requiring sheet music reading skills. We project a graphical notation on top of a piano that indicates what key(s) need to be pressed and create a feedback loop that monitors the player's performance. We implemented The Augmented Piano (TAP), which is a straightforward combination of a physical piano with our alternative notation projected on top. Piano Attack (PAT) extends TAP with a shooting game that continuously provides game-based incentives for learning to play the piano.

Smartphones leak personal information about their owner when they use it to connect to the Internet. Despite recent coverage of these issues in popular media, raising awareness remains problematic since it remains largely invisible to the users. We designed a system, SASQUATCH, consisting of a network scanner and a public display, to draw the visitor's attention and inform them about these issues. SASQUATCH first gathers private information about previous whereabouts, and then shows an anonymized version of this data on the public display to draw the visitor's attention. Next, SASQUATCH offers an interactive component that allows people to view the information their own smartphone is leaking in private, and then provides solutions (including a fully-automated smartphone application) for securing against future privacy leaks. A set of initial field trails has shown that SASQUATCH is highly effective in raising awareness.

Storyboarding is a technique that is often used for the conception of new interactive systems. A storyboard illustrates graphically how a system is used by its users and what a typical context of usage is. Although the informal notation of a storyboard stimulates creativity, and makes them easy to understand for everyone, it is more difficult to integrate in further steps in the engineering process. We present an approach, "Time In Storyboards" (Timisto), to extract valuable information on how various interactions with the system are positioned in time with respect to each other. Timisto does not interfere with the creative process of storyboarding, but maximizes the structured information about time that can be deduced from a storyboard. Design Methods;Analysis Methods

There is a diversity of ways to determine a user's location in a pervasive environment today. On a large scale, this diversity often results in variability of location tracking conditions throughout the environment. For an important class of pervasive applications, which often rely on the ubiquitous availability of location tracking – location-based pervasive applications, the consistency of their behaviour under this variability cannot be guaranteed. This type of limitation raises a need for the adaptation of the application's behaviour that would reflect this variability. We investigated empirically how visualising different aspects of uncertainty about location and of the behaviour of localisation systems affects users’ impressions about a location-based application. The two components – an ontology that models properties of localisation systems and a set of mapping rules that define how these properties should be visualised in a user interface – are at the core of our approach to providing awareness. The results of the investigation show that the additional visual demand, intended for raising users’ awareness of uncertainty about their location tracking conditions, is perceived to be beneficial by users. We also reveal that different characteristics of this awareness are of different importance to users. Furthermore, we conclude that the particular importance depends on users’ personal profiles (e.g. their eyesight level), on the distance between the users (e.g. knowing about someone else's state is less important if they are far) and on the quality of tracking (the importance increases in problematic areas). On the basis of the obtained results and observations, we suggest a set of guidelines and visualisations, which could be used in designing pervasive applications that require location tracking.

Localisation has become a standard feature in many mobile applications. Numerous techniques for both indoor and outdoor location tracking are available today, providing a diversity of ways positioning information can be delivered to a mobile application (e.g., a location-based service). Such factors as the variation of precision over time and covered areas or the difference in quality and reliability make the adoption of several techniques for one application cumbersome. This work presents an approach that models the capabilities of localisation systems and then uses this model to build a unified view on localisation, with special attention paid to uncertainty coming from different localisation conditions and its presentation to the user. We discuss technical considerations, challenges and issues of the approach and report about a user study on users' acceptance of the suggested behaviour of an application based on the approach. The results of the study showed the feasibility of the approach and revealed users' preference towards automatic but yet informed changes they experienced while using the application.

Multi-touch gained a lot of interest in the last couple of years and the increased availability of multi-touch enabled hardware boosted its development. However, the current diversity of hardware, toolkits, and tools for creating multi-touch interfaces has its downsides: there is only little reusable material and no generally accepted body of knowledge when it comes to the development of multi-touch interfaces. This workshop is the second workshop on this topic and the workshop goal remains unchanged: to seek a consensus on methods, approaches, toolkits, and tools that aid in the engineering of multi-touch interfaces and transcend the differences in available platforms. The patterns mentioned in the title indicate that we are aiming to create a reusable body of knowledge.

Pervasive applications are typically realized through ad-hoc service and user interface compositions. While many tools focus on the development of pervasive services by masking the complex technical side of a pervasive computing environment, the deployment of an application as a whole -- i.e. a set of services and user interfaces -- is often forgotten. We present an alternative design strategy and tool for pervasive applications in which pervasiveness is not considered a handicap, but rather as a situation that draws extra attention to the deployment of applications. By crafting pervasive applications and their services as independent context consumers and producers, we illustrate how the behaviour of a pervasive application deployed using our approach can be observed while it executes.

Multi-touch gained a lot of interest in the last couple of years and the increased availability of multi-touch enabled hardware boosted its development. However, the current diversity of hardware, toolkits, and tools for creating multi-touch interfaces has its downsides: there is only little reusable material and no generally accepted body of knowledge when it comes to the development of multi-touch interfaces. This workshop seeks a consensus on methods, approaches, toolkits, and tools that aid in the engineering of multi-touch interfaces and transcend the differences in available platforms. The patterns mentioned in the title indicate that we are aiming to create a reusable body of knowledge.

Efficient delivery of information in vehicular networks is crucial for the creation of useful and usable applications that need to cope with nomadic large-scale environments. Context-awareness is often key to improve efficiency of a vehicle network since it allows to make informed decisions on the data routing, data locality and data necessity for different moving objects. In this paper we show how the social network of vehicle residents, as part of the overall context, allows us to improve the information sharing in the vehicular network significantly. We demonstrate this by deploying a social ubiquitous-help-system (UHS) on top of a vehicular network. We analyze how UHS operates in a vehicular network using a network simulation of realistic large scale vehicular movement data and show that the social interactions increases the efficiency, relevance and quality of information in data delivery. ispartof: pages:1188-1194 ispartof: Proceedings of the 2010 ACM Symposium on Applied Computing pages:1188-1194 ispartof: SAC location:Sierre date:22 Mar - 26 Mar 2010 status: published

We present an approach to exploit social and spatio-temporal context in order to improve information dissemination in dynamic largescale public spaces. We illustrate it by applying a proposed measure of geo-social relevance of each individual in a simulated vehicular network and by comparing the performance of different network message passing techniques in an inter-vehicle ‘help-me-best-and-do-it-fast’ communication scenario. We conclude that the use of social networking capabilities of an individual combined with knowledge about their spatio-temporal context information significantly improves purposeful interaction between individuals in terms of both the efficiency of the network data dissemination and the quality of the delivered information. ispartof: pages:107-116 ispartof: LNCS vol:6439 issue:6439 pages:107-116 ispartof: First International Joint Conference on Ambient Intelligence (AmI'10) location:Malaga, Spain date:10 Nov - 12 Nov 2010 status: published

Storyboards are used in user-centred design (UCD) to clarify a scenario that describes the future use of a system. Although there are many styles of storyboarding, the graphical notation and language are very accessible for all team members of a multidisciplinary team. This papers describes how principles and techniques from comics can facilitate storyboarding in our COMuICSer approach and tool. COMuICSer formalises the way that storyboards are created, while preserving creative aspects of storyboarding. In combination with tool support for COMuICSer, this simplifies the relation of storyboards with other artefacts created in UCD such as structured models and UI designs and supports communication in multidisciplinary teams.

Due to the large amount of mobile devices that continue to appear on the consumer market, mobile user interface design becomes increasingly important. The major issue with many existing mobile user interface design approaches is the time and effort that is needed to deploy a user interface design to the target device. In order to address this issue, we propose the plug-and-design tool that relies on a continuous multi-device mouse pointer to design user interfaces directly on the mobile target device. This will shorten iteration time since designers can continuously test and validate each design action they take. Using our approach, designers can empirically learn the specialities of a target device which will help them while creating user interfaces for devices they are not familiar with.

Discovering and unlocking the full potential of complex pervasive environments is still approached in application-centric ways. A set of statically deployed applications often defines the possible interactions within the environment. However, the increasing dynamics of such environments require a more versatile and generic approach which allows the end-user to inspect, configure and control the overall behavior of such an environment. A meta-UI addresses these needs by providing the end-user with an interactive view on a physical or virtual environment which can then be observed and manipulated at runtime. The meta-UI bridges the gap between the resource providers and the end-users by abstracting a resource's features as executable activities that can be assembled at runtime to reach a common goal. In order to allow software services to automatically integrate with a pervasive computing environment, the minimal requirements of the environment's meta-UI must be identified and agreed on. In this paper we present Meta-STUD, a goal- and service-oriented reference framework that supports the creation of meta-UIs for usage in pervasive environments. The framework is validated using two independent implementation approaches designed with different technologies and focuses.

Pervasive environments can be hard to configure and interact with using handheld computing devices, due to the mismatch between physical and digital worlds. Usually, smart resources in the user's vicinity are discovered and presented in a menu on the user's device from where they can be accessed. However, in environments with many embedded resources it becomes hard to identify resources by means of a textual description and to get aware of the tasks they support. As an alternative to menu-driven interfaces, we demonstrate annotated photos as a means for controlling a pervasive environment. We present as part of our approach a tool that enables people to picture their own environment and use photos as building blocks to create an interactive digital view on their surroundings. To demonstrate and evaluate our approach, we engineered a pervasive prototype application that is operated through a photo-based user interface and assembled using ontologies.

The knowledge of spatial arrangements of objects is an important component for the design of migratable user interfaces that target pervasive environments. Objects in these environments are often moving around individually, which leads to a highly dynamic and unpredictable environment. Due to its nature, spatial information cannot be described exhaustively, and uncertainty and imprecision need to be taken into account during both the design phase and at runtime. We present an approach to model dynamic spatial information, providing it with the ability to interpret to some extent uncertain and imprecise knowledge. We then integrate this type of spatial-awareness into ReWiRe , a framework for designing interactive pervasive environments, in order to improve its user-interface distribution techniques.

Multi-touch interaction techniques are becoming more widespread because of new industrial initiatives to make this hardware available and affordable for the consumer market. To cope with the diversity in hardware set-ups and the lack of knowledge about developing generic multi-touch applications, we created a framework, Eunomia, for abstracting the hardware from the software and to enable software developers to easily develop interactive applications taking advantage of multi-touch interaction. We describe our first set of applications created on top of this framework that are targeted for public spaces. During the deployment of these applications, we were able to observe users that are confronted with multi-touch technologies in a public space.

Multi-touch large display interfaces are becoming increasingly popular in public spaces. These spaces impose specific requirements on the accessibility of the user interfaces: most users are not familiar with the interface and expectations with regard to user experience are very high. Multi-touch interaction beyond the traditional move-rotate-scale interactions is often unknown to the public and can become exceedingly complex. We introduce TouchGhosts: visual guides that are embedded in the multi-touch user interface and that demonstrate the available interactions to the user. TouchGhosts are activated while using an interface, providing guidance on the fly and within the context-of-use. Our approach allows to define reconfigurable strategies to decide how or when a TouchGhost should be activated and which particular visualization will be presented to the user.

The use of handheld computers as a tool to enrich the museum experience has found its way into many museums, opening up new possibilities to increase the attractiveness of museum visits, especially for youngsters. We developed a mobile guide framework that supports the creation of mobile guides and adheres to social-constructivist principles of learning. The mobile museum game we created with this framework aims at developing the social, cognitive and technical skills of the users. Large scale evaluations show the potential power of our approach to stimulate interaction between youngsters and to create a playful learning experience.

In this paper we introduce MuiCSer, a conceptual process framework for Multi-disciplinary User-centred Software Engineering (UCSE) processes. UCSE processes strive for the combination of basic principles and practices from software engineering and user-centred design approaches in order to increase the overall user experience with the resulting product. The MuiCSer framework aims to provide a common understanding of important components and associated activities of UCSE processes. As such, the conceptual framework acts as a frame of reference for future research regarding various aspects and concepts related to this kind of processes, including models, development artefacts and tools. We present the MuiCSer process framework and illustrate its instantiation in customized processes for the (re)design of a system. The conceptual framework has been helpful to investigate the role of members of a multi-disciplinary team when realizing artefacts in a model-based approach. In particular process coverage of existing artefact transformation tools has been studied.

Mobile digital photography has become a ubiquitous service: nearly every mobile phone is equipped with a camera. Huge collections of pictures are created with these phones that are hard to structure in a. usable way. Problems in saving, managing and exchanging digital pictures are becoming typical obstacles that need to be tackled in order to have a usable system. We present PhotoFOAF: a new approach to mobile imaging that exploits the social context of a picture as an important part of the metadata. We present an approach that allows users to insert social metadata in pictures while capturing them. The resulting collections of captured pictures can be managed, searched and exchanged more easily by taking benefit of the personal social relevance of a picture. PhotoFOAF implements an end-to-end process to accomplish this and it exploits, enhances and stimulates social relationships between users during all the different steps.

In this paper, a method for inference of high-level semantic information for image annotation and retrieval is proposed. Bayesian theory is used as a tool to model a belief network to configure semantic labels for image regions. These semantic labels for regions are obtained from a multi visual feature-based object detection approach. The aim is to model potential semantic descriptions of basic objects in the images, the dependencies between them, and the conditional probabilities involved in those dependencies. This information is then used to calculate the probabilities of the effects that those objects have on each other in order to obtain more precise and meaningful semantic labels for the whole images. However, the proposed method is not restricted to the specific region-based approach used in this paper. Rather, the proposed method can be applied in any region-based image retrieval systems. Selected experimental results are presented to show the improved retrieval performance of the proposed method.

The diversity of end-user devices in combination with a growing user base poses important challenges for providing easy access to the huge amount of content and services currently available. Each device has its typical set of capabilities and characteristics that must be taken into account to create an appropriate user interface that provides interactive access to multimedia data and services. Furthermore, end-users also have their specific requirements that influence the accessibility of data and services for individual access. The approach we present in this paper is geared towards the idea of universal access to interactive multimedia data and services for everyone, independent of the user characteristics or end-user device capabilities. For this purpose we combine user and device models with high-level user interface description languages in order to decouple the interface presentation from its platform, and to generate the most suitable interface on a per-user, per- device basis making use of the semantics that are provided by user and device profile.

This paper presents how ad-hoc co-located collaborations can be supported with an arbitrary number of users that only have access to small-size mobile displays. Our approach is based on tracking of these personal displays that share the same information space. All displays involved in the collaboration act as autonomous windows on a set of data items (the information space) positioned on a shared virtual canvas. Data items are identified by their three-dimensional location in physical space and can be manipulated through the displays that serve as windows on the shared canvas. Each display is tracked in physical space and is aware of its own location. Since different mobile displays can access and manipulate the same information space, a distributed locking mechanism makes sure the data stays consistent during simultaneous access of data in this information space.

Semantic service descriptions have paved the way for flexible interaction with services in a mobile computing environment. Services can be automatically discovered, invoked and even composed. On the contrary, the user interfaces for interacting with these services are often still designed by hand. This approach poses a serious threat to the overall flexibility of the system. To make the user interface design process scale, it should be automated as much as possible. We propose to augment service descriptions with high-level user interface models to support automatic user interface adaptation. Our method builds upon OWL-S, an ontology for Semantic Web Services, by connecting a collection of OWL-S services to a hierarchical task structure and selected presentation information. This allows end-users to interact with services on a variety of platforms.

Web services are todays preferred vehicle for creating service-oriented architectures (SOA). Due to the boom of personal networked devices, the Web also found its way to the mobile space. This gives rise to device hosted mobile services that deliver functionality over a wireless network. There is potential to unite these services in a collaborative service space and exploit them as building blocks for distributed applications. This paper concentrates on a light-weight middleware layer to facilitate the integration, accessibility and orchestration of Web-based services in heterogeneous environments. Leveraging Web technology, this layer allows to interconnect serverless, peer-to-peer services deployed across federations of devices. We present its value in a ubiquitous application.

With the shift of networked software applications away from the desktop computers and into home appliances comes the challenge of finding new ways to model this type of software. The most important home appliance that has new found computing capabilities is the television set. Through television people can now be participants instead of viewers, and use interactive software to enter the participative stage. Staged participatory multimedia events are a subset of this kind of interactive software that have a predefined temporal structure that lets television viewers and mobile users become participants in a distributed multimedia event. In this paper, we introduce an interaction model that is part of modelling language for staged participatory multimedia events. We show how the language can be mapped on UML constructs and shortly discuss related work using a common example.

Mobile digital photography has become a ubiquitous service: nowadays, nearly every mobile phone is equipped with a camera. This results in huge collections of pictures that are hard to structure and has lead to problems in saving, managing and exchanging digital pictures. Photo- FOAF presents a new approach to mobile imaging, exploiting the social context of a picture as an important part of the metadata. We present an approach that allows users to insert social metadata in pictures while capturing them. The resulting collections of captured pictures can be managed, searched and exchanged more easily by taking benefit of the personal social relevance of a picture. PhotoFOAF implements an end-to-end process to accomplish this and it exploits, enhances and stimulates social relationships between users during all the different steps.

In this paper we report on our work to enable "laid-back" social interactions using television as a primary interaction medium and mobile devices that participate as a secondary medium. By integrating semantic web techniques with interactive television we were able to create smart applications that can run as extensions of television shows and stimulate groups of users to communicate. Participants are teamed up to play along with a television show. Teams are composed based on common ground, which can be shared interests as well as shared characteristics that can be found between the users. Participating is possible using a set-top box or a mobile device; this allows people that are co-located and share a television set to participate individually. The usage of a mobile device to participate does not require a television set however: users on the move can also play along in a team. Our system does not require a new television format, but is also able to reuse existing television shows and socialize them in order to be re-broadcasted with support for distributed teams that play along with the show.

In a firefighter emergency mission it is essential for the members of a fire brigade to get an intelligent and reliable overview of the complete situation, presented according to the role of each member In this paper we report on the design and development of a system to support a fire brigade on site with a set of mobile services that offers a role-based focus+context user interface It provides the required overview over the emergency situation according to the user task and context, while life-saving information is emphasized The implementation of a context-rule-based decision module enhances the visualization of required information Interaction with the user interface is designed for use in the wild; which in this case comes down to providing a “fat finger” interface that allows firemen to interact with the user interface on site with his gloves on.

In this paper we report on our work to enable "laid-back" social interactions using television as a primary interaction medium. By integrating semantic web techniques with interactive television we were able to create smart applications that can run as extensions of television shows and stimulate groups of users to communicate. Groups are based on the shared characteristics that can be found for subsets of spectators. Communication between spectators is brought about at two levels: direct communication like instant messaging and indirect communication like cooperating in a team to win a quiz. Our system does not necessarily require a new television format, but is able to reuse existing television shows and to "socialize" them so they can be re-broadcasted with support for group interaction.

The wide diversity of consumer devices has led to new methodologies and techniques to make digital content available over a broad range of devices with minimal effort. In particular the design of the interactive parts of a system has been the subject of a lot of research efforts because these parts are the most visible and are critical for the usability (and thus use) of a system. One thing that is missing in many current approaches is the ability to combine these new methodologies and techniques with a user-centric approach to ensure preferences from and requirements for a specific user are taken into account besides the device adaptations. In this paper we analysed the applicability of MPEG-21, part 7: Digital Item Adaptation, for the adaptation of a user interface to user characteristics. We show how the high-level XML-based user interface description language UIML in combination with an MPEG-21-based user profile enables designers to create accessible and personalised multi-device user interfaces. Using this combination results in user interfaces that can be deployed on a broad range of devices while taking into account user preferences with minimal effort. This approach enhances accessibility to digital items on various platforms, since all interactions with digital items should be supported by an appropriate user interface.

As the diversity of available computing devices increases it becomes more difficult to adapt User Interface development to support the full range of available devices. One of the difficulties are the different GUI libraries: to use an alternative library or device one is forced to redevelop the interface completely for the alternative GUI library. To overcome these problems the User Interface Mark-up Language (UIML) specification has been proposed, as a way of glueing the interface design to different GUI libraries in different environments without further efforts. In contrast with other approaches UIML has matured and has some implementations proving its usefulness. We introduce the first UIML renderer for the .Net framework, a framework that can be accessed by different kinds of programming languages and can use different kinds of widget sets. We show that its properties, among them its reflection mechanism, are suitable for the development of a reusable and portable UIML renderer. The suitability for multi-device rendering is discussed in comparison with our own multi-device UI framework Dygimes. The focus is on how layout management can be generalised in the specification to allow the GUI to adapt to different screen sizes.

In this paper we show an approach that allows web interfaces to be dynamically distributed among several interconnected heterogeneous devices in an environment to support the tasks and activities the user performs. The approach uses a light-weight HTTP-based daemon as a distribution manager and RelaxNG schemas to describe the service user interfaces offered by native applications. From these service descriptions, the XHTML-based user interface is generated.