Search

Over the decades, technologies envisioned by pioneers such as Douglas Engelbart are becoming a reality. AI has become an important driver for technological progress, posing questions for the future of human-computer interaction. Our research group at Fraunhofer FIT looks back on a 51 year long research tradition that started with Engelbart’s vision and followed through the larger developments of HCI from the introduction of CSCW up to successful projects involving the engineering of large software systems in practice. In this paper, we outline the history of our institute against the background of trends in HCI, working out the cornerstones and “useful practices” from our research tradition towards our vision of the future of a human-centred AI/IA, with the expectation that this analysis may be useful for similar organisations. In doing so, we illustrate tensions between theory and application, humans and technology, and show how keeping those aspects in balance is an important challenge and chance for bringing disruptive emerging technologies successfully into practice.

In this article, we compare a Google Tango tablet with the Microsoft HoloLens smartglasses in the context of the visualisation and interaction with Building Information Modeling data. A user test was conducted where 16 participants solved four tasks, two for each device, in small teams of two. Two aspects are analysed in the user test: the visualisation of interior designs and the visualisation of Building Information Modeling data. The results show that the Tango tablet is surprisingly preferred by most users when it comes to collaboration and discussion in our scenario. While the HoloLens offers hands-free operation and a stable tracking, users mentioned that the interaction with the Tango tablet felt more natural. In addition, users reported that it was easier to get an overall impression with the Tango tablet rather than with the HoloLens smartglasses.

We present a new SLAM system capable of producing high quality globally consistent surface reconstructions with accurate real-time tracking and localization abilities. The system works on an off the shelf laptop with a typical GPU. This paper proposes an approach to unify feature-based keyframe techniques with fused volumetric surface reconstruction methods to overcome both of their limitations. On one hand, feature-based keyframe SLAM techniques have reached a level of maturity and can guarantee accurate and real-time tracking and localization ability, but their raw RGB-D point clouds are too noisy. On the other hand, volumetric surface reconstruction methods can produce a dense surface reconstruction of the environment, which will be helpful for Augmented Reality (AR) applications and scene understanding. However, current dense SLAM systems have limited tracking ability, which is vital for the quality of surface reconstruction. Moreover most of the current dense SLAM systems have to run on a powerful desktop PC to guarantee realtime performance. By unifying the feature-based keyframe tracking ability and adopting a multi-threaded design, our system improves both the tracking ability and the real-time performance. We present results of a wide variety of aspects of our system and evaluate it using the widely used TUM RGB-D and ICL-NUIM Datasets. Our system achieves unprecedented performance in terms of trajectory estimation, surface reconstruction, real-time and computational performance in comparison to other start-ofthe-art dense RGB-D SLAM systems.

The advent of Building Information Modelling (BIM) provides geometry data that can be easily used for visualisations. We present six demonstrators made from the same data using similar workflows. They cover different categories of mobile devices, ranging from head-mounted displays to smartphones and tablets with inside-out positional tracking. They showcase cross-media visualisations depending on the device capabilities, which vary from a sophisticated car-based AR-setup, over wired and wireless VR, to see-through AR on smart glasses, and video-based AR on tablets.

Touring location-based experiences is challenging, as both content and underlying location services must be adapted to each new setting. A study of a touring performance called Rider Spoke as it visited three different cities reveals how professional artists developed a novel approach to these challenges in which users drove the co-evolution of content and the underlying location service as they explored each new city. We show how the artists iteratively developed filtering, survey, visualization, and simulation tools and processes to enable them to tune the experience to the local characteristics of each city. Our study reveals how by paying attention to both content and infrastructure issues in tandem, the artists were able to create a powerful user experience that has since toured to many different cities.

This paper presents a user-study of the TOTEM.Scout mobile tool that has been developed for in-situ creation of location-based content for games and edutainment applications based on configurable data templates. It works in cooperation with the TOTEM.Designer web-based tool and enables convenient structuring, management and export of the created content to prepare arbitrary location-based experiences. We motivate the necessity of this tool as well as its architecture, and describe the layout, results and findings of the performed case study in a real context of use. The results provide insights into possible improvements in terms of user interface and the tool's functionality design, and suggest new features for data collection and management.

This paper reports on AREEF, the world's first multi-player Under Water Augmented Reality (UWAR) experience. The underlying mission of this work was to bring computer games and entertainment applications from traditional settings into the water using Augmented Reality (AR) technology. We provide an application overview and present findings from our participatory design process that involved engineers and designers, as wells as end-users and water-park experts. The paper closes with a brief discussion of technical aspects that relate to Wi-Fi communication and computer vision tracking, and provides an outlook for future work.

This paper presents the location-based game Tidy City and the accompanying authoring tools that enable users to create new scavenger hunt like missions. In the game the players need to physically explore their city by interpreting clues to find the correct target destination. The in-situ authoring tool Tidy City Scout enables designers to likewise walk around the city and collecting notes, images and GPS data for potential riddles. The riddles are finalized in a web-based authoring tool and then published to all players of the game. The game and its tools are freely accessible for non-commercial purposes at http://totem.fit.fraunhofer.de/tidycity.

Touring location-based experiences is challenging as both content and underlying location-services must be adapted to each new setting. A study of a touring performance called Rider Spoke as it visited three different cities reveals how professional artists developed a novel approach to these challenges in which users drove the co-evolution of content and the underlying location-service as they explored each new city. We show how the artists iteratively developed filtering, survey, visualization and simulation tools and processes to enable them to tune the experience to the local characteristics of each city. Our study reveals how by paying attention to both content and infrastructure issues in tandem the artists were able to create a powerful user experience that has since toured to many different cities.

In the gaming area, augmented reality (AR) and especially mobile augmented reality provides unique opportunities. Unlike traditional video games, mobile augmented reality games are not imprisoned in the screen-space but provide for interaction with the world that surrounds us. They incorporate real locations and objects into the game, therefore tapping into a set of pre-existing thoughts, emotions and real-life experiences of its players, which in turn provides the material for a much richer gaming world and user experience. In mobile augmented reality games the playing area becomes borderless and they can be played literally anywhere and anytime. The current advancement of modern cell phone technology is finally giving more people than ever the hardware necessary to participate and experience such games.

This chapter (Boyd Davis principal author, project director) discusses issues arising from the development of a world-first location-aware game (’Ere be Dragons) using live bio-sensing, funded by the Wellcome Trust and supported by Hewlett Packard Research, who created the entirely new combinations of technology used for the project. The chapter contributes to the understanding of ‘serious games’, a field traceable to George Mallen’s 1968 Eco Game but increasingly significant in the past decade. The project used imagery and animation to represent combined data gathered from two live streams, location and heart-rate, exploring the new possibilities of media that are pervasive, multimodal and physical. It previously received international attention and awards reported in RAE2008; the new contribution here was to identify the relevance for medical information technology. This invited chapter was the only one in the book not written by medical experts. The research programme required a cross-disciplinary team of artists, interaction experts and health scientists, using art to explore bioscience in the context of public health; the science collaborator was Professor Riddoch, Senior Scientific Editor, Chief Medical Officer's report, Physical Activity and Health (2004). It combined novel mobile technologies with novel gameplay, unlike most digital games, which are reactionary and unimaginative in their models of play. Finally, formal and informal evaluation was undertaken with users. The chapter focuses on the use of digital gameplay as a means of modifying attitudes and behaviours, and the benefits of live feedback during use as distinct from retrospective review of performance. Two contexts are described: the crisis in health caused by widespread low levels of physical activity and the techno-cultural context in which the work was developed. Few projects have been based on live heart-rate, and the chapter explains the problems and the potential of exploiting this aspect of behaviour in digital play.

In this chapter we provide an overview of the area of mobile location-based gaming and its relation to maps. Digital maps of various forms are essential to enable the implementation of mobile location-based games and support various key tasks from content authoring and interaction scripting over game state management and location-based interaction at run-time, to content presentation and user orientation. Based on an analysis of early experiments and currently existing systems we examine requirements with regards to hardware, software, standards and game design and discuss their implications for future developments.

In current authoring tools for location-aware applications the designer typically places trigger zones onto a map of the target environment and associates these with events and media assets. However, studies of deployed experiences have shown that the characteristics of the usually invisible ubiquitous computing infrastructure, especially limited coverage and accuracy, have a major impact on an experience. We propose a new approach in which designers work with three layers of information: information about the physical world, information about digital media, but also visualizations of ubiquitous infrastructure. We describe the implementation of a prototype authoring tool that embodies this approach and describe how it has been used to author a location-based game for mobile phones called Tycoon. We then outline the key challenges involved in generalizing this approach to more powerful authoring tools including acquiring and visualizing infrastructure data, acquiring map data, and flexibly specifying how digital content relates to both of these.

A 3D information space is a three-dimensional visualization that contains 2D visualizations and puts them in a semantic context. In this poster, we sketch how Mixed Reality (MR) can be exploited as a technology in order to realize better interaction in a 3D information space and, as a result, to develop new interactive visualization techniques. Here, the real space is used as a metaphor for interacting with the 3D information space.