Search

We present an Augmented Reality (AR) application intended for use in supermarkets, with the primary goal to bring fun and digital engagement through AR mini-games to the customers while shopping. We believe that our approach can be extended and scaled up by integrating mini-games into existing shopping apps in the future.

Over the decades, technologies envisioned by pioneers such as Douglas Engelbart are becoming a reality. AI has become an important driver for technological progress, posing questions for the future of human-computer interaction. Our research group at Fraunhofer FIT looks back on a 51 year long research tradition that started with Engelbart’s vision and followed through the larger developments of HCI from the introduction of CSCW up to successful projects involving the engineering of large software systems in practice. In this paper, we outline the history of our institute against the background of trends in HCI, working out the cornerstones and “useful practices” from our research tradition towards our vision of the future of a human-centred AI/IA, with the expectation that this analysis may be useful for similar organisations. In doing so, we illustrate tensions between theory and application, humans and technology, and show how keeping those aspects in balance is an important challenge and chance for bringing disruptive emerging technologies successfully into practice.

This paper presents the concept, a system-overview, and the evaluation of EnArgus, the central information system for energy research funding in Germany. Initiated by the German Federal Ministry for Economic Affairs and Energy (BMWi), EnArgus establishes a one-stop information system about all recent and ongoing energy research funding projects in Germany. Participants ranging from laypersons to experts were surveyed in three workshops to evaluate both the public and expert interfaces of the EnArgus system in comparison to peer systems. The results showed that the EnArgus system was predominantly evaluated positively by the various participants. It contributes to making the energy sector more transparent and offers clear advantages for professional use compared to similar systems. The system’s semantic processing enables more precise hits and better coverage by including semantically related terms in search results; its intelligence makes it fail-safe, rendering it suitable for areas where poor results can have dire consequences. Reporting on an actual real-world system, the paper also provides a roadmap-view of how electronic filing of administrative project data can be semantically enhanced and opened-up to provide the basis for new ways into the data that are key for future breakthrough AI interfaces.

The presented demonstrator showcases the potential of augmented reality (AR) smartglasses in the sterile supply chain. In the scope of the project "Smartglasses in der Sterilgutver-sorgung", we developed an application to investigate on the applicability of smartglasses and AR paradigms during the reprocessing of used operational tools. The demonstrator was developed and tested using the iterative user-centered approach involving actual employees of the respective domain.

In this article, we compare a Google Tango tablet with the Microsoft HoloLens smartglasses in the context of the visualisation and interaction with Building Information Modeling data. A user test was conducted where 16 participants solved four tasks, two for each device, in small teams of two. Two aspects are analysed in the user test: the visualisation of interior designs and the visualisation of Building Information Modeling data. The results show that the Tango tablet is surprisingly preferred by most users when it comes to collaboration and discussion in our scenario. While the HoloLens offers hands-free operation and a stable tracking, users mentioned that the interaction with the Tango tablet felt more natural. In addition, users reported that it was easier to get an overall impression with the Tango tablet rather than with the HoloLens smartglasses.

The idea of augmented or virtual reality in combination with head mounted display is being discussed already since at least 1968. However, for a long time, this topic was discussed mainly within the academic research area with only limited effect or uptake in the work place. Primary reason for this was the missing availability of robust and affordable hardware as well as the limited mobile graphics capabilities. This has changed recently with the availability of numerous affordable devices in combination with applications from the entertainment and gaming area. This Special Issue on Smart Glasses presents a mix of recent research papers and reports to provide an overview of ongoing research and developments in work place environments. In the remainder of this introductory paper we present an overview of the history of Smart Glasses and their applications over the last decades. We also clarify the term Augmented Reality in this historic context. Then we present a topology of current products as well as their intended application areas. Finally, we introduce the papers of this issue within this context.

Educational location-based games provide a link between content and its real-life relevance in a physical environment. Location-based activities for authentic learning provide multiple opportunities, but educators still perceive technological and organisational barriers. There is a need for easy-to-use tools to facilitate the design of playful location-based mobile learning activities that can be integrated into larger curriculums. In this project a transdisciplinary team (educational experts in outdoor education, in nutrition and consumer education, computer scientists) co-created an online authoring system for location-based games, the MILE.designer. This authoring system provides several formats of tasks that can easily be adapted and located intuitively using a simple map interface. Several tasks can be combined into an educational geogame to be provided for a native smartphone app, the MILE.explorer. The theoretical background and the transdisciplinary development process are described, formative and summative evaluation results based on participatory observation and on focus group discussions are presented and further implications discussed.

We present a new SLAM system capable of producing high quality globally consistent surface reconstructions with accurate real-time tracking and localization abilities. The system works on an off the shelf laptop with a typical GPU. This paper proposes an approach to unify feature-based keyframe techniques with fused volumetric surface reconstruction methods to overcome both of their limitations. On one hand, feature-based keyframe SLAM techniques have reached a level of maturity and can guarantee accurate and real-time tracking and localization ability, but their raw RGB-D point clouds are too noisy. On the other hand, volumetric surface reconstruction methods can produce a dense surface reconstruction of the environment, which will be helpful for Augmented Reality (AR) applications and scene understanding. However, current dense SLAM systems have limited tracking ability, which is vital for the quality of surface reconstruction. Moreover most of the current dense SLAM systems have to run on a powerful desktop PC to guarantee realtime performance. By unifying the feature-based keyframe tracking ability and adopting a multi-threaded design, our system improves both the tracking ability and the real-time performance. We present results of a wide variety of aspects of our system and evaluate it using the widely used TUM RGB-D and ICL-NUIM Datasets. Our system achieves unprecedented performance in terms of trajectory estimation, surface reconstruction, real-time and computational performance in comparison to other start-ofthe-art dense RGB-D SLAM systems.

Zusammenfassung Im MILE-Projekt werden digitale Technologien fur mobile, ortsbezogene Lernangebote in der Ernahrungs- und Verbraucherbildung (EVB) genutzt. Ziel ist es, mit sog. MILEGeogames Jugendliche fur Themen der EVB zu sensibilisieren. Die im Projekt entwickelte MILE.Toolbox unterstutzt Multiplikatoren bei der Erstellung von MILE-Geogames. Der Beitrag diskutiert das Potential des Ansatzes fur die Ernahrungs- und Verbraucherbildung. Schlusselworter: Mobiles, ortsbezogenes Lernen, Geogames, Ernahrungs- und Verbraucherbildung ----- Bibliographie: Muller, Heike/Bartsch, Silke/Eisenhardt, Manuela/Oppermann, Leif/Schaal, Steffen: Mobiles, ortsbezogenes Lernen in der Ernahrungs- und Verbraucherbildung, HiBiFo, 2-2017, S. 78-90. https://doi.org/10.3224/hibifo.v6i2.07

The advent of Building Information Modelling (BIM) provides geometry data that can be easily used for visualisations. We present six demonstrators made from the same data using similar workflows. They cover different categories of mobile devices, ranging from head-mounted displays to smartphones and tablets with inside-out positional tracking. They showcase cross-media visualisations depending on the device capabilities, which vary from a sophisticated car-based AR-setup, over wired and wireless VR, to see-through AR on smart glasses, and video-based AR on tablets.

This paper reports on a study of AREEF, a multi-player Underwater Augmented Reality (UWAR) experience for swimming pools. Using off-the-shelf components combined with a custom made waterproof case and an innovative game concept, AREEF puts computer game technology to use for recreational and educational purposes in and under water. After an experience overview, we present evidence gained from a user-centred design-process including a pilot study with 3 kids and a final evaluation with 36 kids. Our discussion covers technical findings regarding marker placement, tracking, and device handling, as well as design related issues like virtual object placement and the need for extremely obvious user interaction and feedback when staging a mobile underwater experience.

The Smartphone Project (TSP) is an interactive dance-performance in a professional setting that exploits the communication channels provided by smartphone-apps as a new material in the dance-theatre domain. We present an account of the experience and its staging. Based on an initial study with 36 participants from the audience, we present results and discuss lessons learned from this project that might guide similar future work.

This paper presents a suite for configuring and deploying mobile, location-based educational games for natural history museums. We propose a novel approach that uses structured museum-data as game-content and present an initial study thereof. The overall aim is to enable domain-experts, like teachers or museum educators, to create educational games for off-the-shelf mobile phones without requiring neither bespoke programming skills nor expensive setups. Our approach builds on the combination of two strands of work: 1.) structuring and providing museum-assets as linked data, and 2.) template-based content creation for mobiles games. This is thought to facilitate reuse of existing data and foster its maintenance on the one hand, as well as providing a timely and engaging mobile interface to museum content on the other hand. Our study is based on two rounds of user-tests with think-aloud observations and questionnaires, and on the developers' notes and reflections that stem from introductory workshops at four natural history museums from the Natural Europe project consortium. We found that both, the workflow for content-creation and the mobile end-user app have high hedonic as well as pragmatic qualities. Thus, the test results indicate that our approach might provide guidance for future work in this domain.

This paper presents a user-study of the TOTEM.Scout mobile tool that has been developed for in-situ creation of location-based content for games and edutainment applications based on configurable data templates. It works in cooperation with the TOTEM.Designer web-based tool and enables convenient structuring, management and export of the created content to prepare arbitrary location-based experiences. We motivate the necessity of this tool as well as its architecture, and describe the layout, results and findings of the performed case study in a real context of use. The results provide insights into possible improvements in terms of user interface and the tool's functionality design, and suggest new features for data collection and management.

This paper reports on AREEF, the world's first multi-player Under Water Augmented Reality (UWAR) experience. The underlying mission of this work was to bring computer games and entertainment applications from traditional settings into the water using Augmented Reality (AR) technology. We provide an application overview and present findings from our participatory design process that involved engineers and designers, as wells as end-users and water-park experts. The paper closes with a brief discussion of technical aspects that relate to Wi-Fi communication and computer vision tracking, and provides an outlook for future work.

In Kap. 1 haben wir schon VR als innovative Form der Mensch-Computer-Interaktion kennengelernt. In diesem Kapitel behandeln wir die Gestaltung und Realisierung von Interaktionen und der daraus resultierenden Benutzungsschnittstelle (engl. User Interface ) eines VR-Systems detailliert. Ein Nutzer interagiert mit einem VR-System, um virtuelle Objekte auszuwahlen (Selektion) und zu verandern (Manipulation) sowie seine Position und seine Blickrichtung in der Virtuellen Umgebung zu bestimmen (Navigation). Dazu kommt die Interaktion mit dem VR-System selbst (Systemsteuerung), um auf einer Metaebene Funktionen auserhalb der Virtuellen Umgebung auszufuhren (z. B. das Laden einer neuen Virtuellen Welt). Diese Grundaufgaben der Selektion, Manipulation, Navigation und Systemsteuerung werden in je einem Unterkapitel behandelt. Dabei werden Losungen fur die Realisierung dieser Interaktionen vorgestellt. Wesentlich dabei ist, dass eine gute Usability (dt. Gebrauchstauglichkeit) erreicht wird. Dies ist ein Kern der Mensch-Computer-Interaktion allgemein und wir gehen deshalb auf Grundlagen aus diesem Bereich gleich zu Beginn dieses Kapitels ein. Schlieslich betrachten wir in einem Unterkapitel spezielle Entwurfsprozesse, die einen Entwickler bei der Gestaltung und der Realisierung von VR-Interaktionen leiten. Ein wesentlicher Gesichtspunkt dabei ist das wiederholte Validieren der Interaktionen mit Nutzern in Form von Nutzertests. Methoden fur die Durchfuhrung und Auswertung von Nutzertests werden daher gesondert im letzten Unterkapitel thematisiert.

Part 18: Mobile Gaming, Mobile Life - Interweaving the Virtual and the Real; International audience; Over the last few years we have witnessed the smartphone dominating the market, the rapid growth of mobile apps, a surge in mobile augmented reality and location-based apps, and burgeoning mobile communities. While mobile topics continue to provide rich research challenges, people and companies outside academia already use these apps regularly. This is due to the increased availability of affordable devices, applications and technologies that support the creation of mixed reality experiences. Thus a core theme of our workshop at ICEC 2012 is authoring mixed realities, designing mobile games and creating mobile experiences. We would like to assess how professional designers and developers, as well as academics and end users, are using the technology to connect the digital and the real in a mobile context.

Designing Augmented Reality location aware games requires an understanding of how form and content issues impact on presence. A study of 60 players was conducted using questionnaires, video analysis and interviews. The results indicate that content including: moral dilemmas, strong narratives, using real locations effectively and applying simple physical behavior within virtual characters to improve embodiment have a positive impact on player experience. The results are presented in the form of guidelines.

This paper presents the location-based game Tidy City and the accompanying authoring tools that enable users to create new scavenger hunt like missions. In the game the players need to physically explore their city by interpreting clues to find the correct target destination. The in-situ authoring tool Tidy City Scout enables designers to likewise walk around the city and collecting notes, images and GPS data for potential riddles. The riddles are finalized in a web-based authoring tool and then published to all players of the game. The game and its tools are freely accessible for non-commercial purposes at http://totem.fit.fraunhofer.de/tidycity.

Touring location-based experiences is challenging as both content and underlying location-services must be adapted to each new setting. A study of a touring performance called Rider Spoke as it visited three different cities reveals how professional artists developed a novel approach to these challenges in which users drove the co-evolution of content and the underlying location-service as they explored each new city. We show how the artists iteratively developed filtering, survey, visualization and simulation tools and processes to enable them to tune the experience to the local characteristics of each city. Our study reveals how by paying attention to both content and infrastructure issues in tandem the artists were able to create a powerful user experience that has since toured to many different cities.

In the gaming area, augmented reality (AR) and especially mobile augmented reality provides unique opportunities. Unlike traditional video games, mobile augmented reality games are not imprisoned in the screen-space but provide for interaction with the world that surrounds us. They incorporate real locations and objects into the game, therefore tapping into a set of pre-existing thoughts, emotions and real-life experiences of its players, which in turn provides the material for a much richer gaming world and user experience. In mobile augmented reality games the playing area becomes borderless and they can be played literally anywhere and anytime. The current advancement of modern cell phone technology is finally giving more people than ever the hardware necessary to participate and experience such games.

This chapter (Boyd Davis principal author, project director) discusses issues arising from the development of a world-first location-aware game (’Ere be Dragons) using live bio-sensing, funded by the Wellcome Trust and supported by Hewlett Packard Research, who created the entirely new combinations of technology used for the project. The chapter contributes to the understanding of ‘serious games’, a field traceable to George Mallen’s 1968 Eco Game but increasingly significant in the past decade. The project used imagery and animation to represent combined data gathered from two live streams, location and heart-rate, exploring the new possibilities of media that are pervasive, multimodal and physical. It previously received international attention and awards reported in RAE2008; the new contribution here was to identify the relevance for medical information technology. This invited chapter was the only one in the book not written by medical experts. The research programme required a cross-disciplinary team of artists, interaction experts and health scientists, using art to explore bioscience in the context of public health; the science collaborator was Professor Riddoch, Senior Scientific Editor, Chief Medical Officer's report, Physical Activity and Health (2004). It combined novel mobile technologies with novel gameplay, unlike most digital games, which are reactionary and unimaginative in their models of play. Finally, formal and informal evaluation was undertaken with users. The chapter focuses on the use of digital gameplay as a means of modifying attitudes and behaviours, and the benefits of live feedback during use as distinct from retrospective review of performance. Two contexts are described: the crisis in health caused by widespread low levels of physical activity and the techno-cultural context in which the work was developed. Few projects have been based on live heart-rate, and the chapter explains the problems and the potential of exploiting this aspect of behaviour in digital play.

Previous research has proposed that authoring tools for location-aware mobile experiences should be extended to reveal the usually hidden ubiquitous computing infrastructure to designers so that they can take account of its characteristics when placing digital content. This paper explores this idea in practice, describing how a suite of such authoring tools has evolved through an iterative process of collaborating with artists to create a location-based game. Reflections on the design process identify the need to support the mobile capture, verification and annotation of mapping data 'in the field' and the visualization and use of this data back 'in the studio' as part of a tightly integrated workflow. We also identify the need for more powerful visualizations that can deal with multiple layers of information representing different locations and networks, and that can move between abstract and detailed views of the data on demand.

A 3D information space is a three-dimensional visualization that contains 2D visualizations and puts them in a semantic context. In this poster, we sketch how Mixed Reality (MR) can be exploited as a technology in order to realize better interaction in a 3D information space and, as a result, to develop new interactive visualization techniques. Here, the real space is used as a metaphor for interacting with the 3D information space.

The user interface and its ease of use are seen as a crucial point for the enduring success of almost any application. The creation of effective user interfaces is no simple task, especially within the domain of augmented reality. Many developers still use traditional desktop-interfaces for their AR applications, even if they are not really suitable for the user, simply because the development of a usable 3D interface is much too complicated and thus consuming too much time and money. This paper describes the ARGUI system, which provides developers of ARToolkit applications with the possibility to create 2D interaction-surfaces registered in 3D. With ARGUI 2D interactions on 2D objects registered in 3D are possible, e.g. attached to a marker. The integration of a complete 2D GUI library is shown in detail, which simplifies the creation of the user interface even more.

This paper charts the distinctive challenges of designing mobile experiences for cyclists and presents two studies of mobile cycle-based experiences: one a heritage tour; the other an exploration of a city at dusk involving recording and listening to personal stories. To understand the cyclists' experiences questionnaires, interviews and observations are drawn on to derive eight lessons for designing cycle-based interaction including: cycling proficiency, physicality, impact of the environment, media and hardware design, collaboration, and safety. The conclusion is that design has to respect the distinctive nature of cycling as a mode of transport and needs to carefully interweave moments of interaction with it.

Augmented/Virtual Reality (AR/VR) is still a fragmented space to design for due to the rapidly evolving hardware, the interdisciplinarity of teams, and a lack of standards and best practices. We interviewed 26 professional AR/VR designers and developers to shed light on their tasks, approaches, tools, and challenges. Based on their work and the artifacts they generated, we found that AR/VR application creators fulfill four roles: concept developers, interaction designers, content authors, and technical developers. One person often incorporates multiple roles and faces a variety of challenges during the design process from the initial contextual analysis to the deployment. From analysis of their tool sets, methods, and artifacts, we describe critical key challenges. Finally, we discuss the importance of prototyping for the communication in AR/VR development teams and highlight design implications for future tools to create a more usable AR/VR tool chain.