Search

In the context of collaborative robotics, handing over hand-held objects to a robot is a safety-critical task. Therefore, a robust distinction between human hands and presented objects in image data is essential to avoid contact with robotic grippers. To be able to develop machine learning methods for solving this problem, we created the OHO (Object Hand-Over) dataset of tools and other everyday objects being held by human hands. Our dataset consists of color, depth, and thermal images with the addition of pose and shape information about the objects in a real-world scenario. Although the focus of this paper is on instance segmentation, our dataset also enables training for different tasks such as 3D pose estimation or shape estimation of objects. For the instance segmentation task, we present a pipeline for automated label generation in point clouds, as well as image data. Through baseline experiments, we show that these labels are suitable for training an instance segmentation to distinguish hands from objects on a per-pixel basis. Moreover, we present qualitative results for applying our trained model in a real-world application.

The idea of using mobile assistance robots for gait training in rehabilitation has been increasingly explored in recent years due to the associated benefits. This paper describes how the previous results of research and praxis on gait training with a mobile assistance robot in orthopedic rehabilitation can be transferred to ophthalmic-related orientation and mobility training for blind and visually impaired people. To this end, the specific requirements for such orientation and mobility training are presented from a therapeutic perspective. Using sensory data, it is investigated how the analysis of training errors can be automated and transferred back to the training person. These pre-examinations are the prerequisite for any form of robot-assisted mobile gait training in ophthamological rehabilitation, which does not exist so far and which is expected to be of great benefit to these patients.

For autonomous mobile service robots, closed doors that are in their way are restricting obstacles. In order to open doors with on-board manipulation skills, a robot needs to be able to localize the door's key features, such as the hinge and handle, as well as the current opening angle. While there are vision-based approaches for detecting doors and handles in images, we concentrate on analyzing 2D laser range scans. This requires less computational effort, and laser-scan sensors are available on most mobile robot platforms. Therefore, we developed three different machine learning approaches and a heuristic method based on line fitting able to extract the required position data. The algorithms are compared with respect to localization accuracy with help of a dataset containing laser range scans of doors. Our LaserDoors dataset is publicly available for academic use. Pros and cons of the individual methods are discussed; basically, the machine learning methods could outperform the heuristic method, but require special training data when applied in a real application.

This paper presents an application of neural networks operating on multimodal 3D data (3D point cloud, RGB, thermal) to effectively and precisely segment human hands and objects held in hand to realize a safe human-robot object handover. We discuss the problems encountered in building a multimodal sensor system, while the focus is on the calibration and alignment of a set of cameras including RGB, thermal, and NIR cameras. We propose the use of a copper-plastic chessboard calibration target with an internal active light source (near-infrared and visible light). By brief heating, the calibration target could be simultaneously and legibly captured by all cameras. Based on the multimodal dataset captured by our sensor system, PointNet, PointNet++, and RandLA-Net are utilized to verify the effectiveness of applying multimodal point cloud data for hand-object segmentation. These networks were trained on various data modes (XYZ, XYZ-T, XYZ-RGB, and XYZ-RGB-T). The experimental results show a significant improvement in the segmentation performance of XYZ-RGB-T (mean Intersection over Union: 82.8% by RandLA-Net) compared with the other three modes (77.3% by XYZ-RGB, 35.7% by XYZ-T, 35.7% by XYZ), in which it is worth mentioning that the Intersection over Union for the single class of hand achieves 92.6%.

In order to meet the increasing demands of mobile service robot applications, a dedicated perception module is an essential requirement for the interaction with users in real-world scenarios. In particular, multi sensor fusion and human re-identification are recognized as active research fronts. Through this paper we contribute to the topic and present a modular detection and tracking system that models position and additional properties of persons in the surroundings of a mobile robot. The proposed system introduces a probability-based data association method that besides the position can incorporate face and color-based appearance features in order to realize a re-identification of persons when tracking gets interrupted. The system combines the results of various state-of-the-art image-based detection systems for person recognition, person identification and attribute estimation. This allows a stable estimate of a mobile robot's user, even in complex, cluttered environments with long-lasting occlusions. In our benchmark, we introduce a new measure for tracking consistency and show the improvements when face and appearance-based re-identification are combined. The tracking system was applied in a real world application with a mobile rehabilitation assistant robot in a public hospital. The estimated states of persons are used for the user-centered navigation behaviors, e.g., guiding or approaching a person, but also for realizing a socially acceptable navigation in public environments.

A successful rehabilitation after surgery in hip endoprosthetics comprises self-training of the lessons taught by physiotherapists. While doing this, immediate feedback to the patient about deviations from physiological gait patterns during training is important. Such immediate feedback also concerns the correct usage of forearm crutches in three-point gait. In the project ROGER, a mobile Socially Assistive Robot (SAR) to support patients after surgery in hip endoprosthetics is going to be developed. The current implementation status of the robotic application developed for the use in a real-world scenario is presented below.

Older people tend to have difficulties using unknown technical devices and are less willing to accept technical shortcomings. Therefore, a robot that is supposed to support older people in managing daily life has to adapt to the users' needs and capabilities that are very heterogeneous within the target group. The aim of the presented case study was to provide in-depth insights on individual usage patterns and acceptance of a mobile service robot in real live environments (i.e. in the users' homes). Results from three cases (users aged 67, 78 and 85 living in their own apartments) are reported. Findings on usability and user experience illustrate that the robot has considerable potential to be accepted to support daily living at home.

We present a new method capable of learning multiple categories in an interactive and life-long learning fashion to approach the "stability-plasticity dilemma". The problem of incremental learning of multiple categories is still largely unsolved. This is especially true for the domain of cognitive robotics, requiring real-time and interactive learning. To achieve the life-long learning ability for a cognitive system, we propose a new learning vector quantization approach combined with a category-specific feature selection method to allow several metrical "views" on the representation space of each individual vector quantization node. These category-specific features are incrementally collected during the learning process, so that a balance between the correction of wrong representations and the stability of acquired knowledge is achieved. We demonstrate our approach for a difficult visual categorization task, where the learning is applied for several complex-shaped objects rotated in depth., (Copyright © 2011 Elsevier Ltd. All rights reserved.)

This paper describes the user-centered design and evaluation process of a humanoid mobile shopping robot named TOOMAS that assists customers of home improvement stores. Three separate empirical field studies addressing the robot's usability (according to ISO 9241-11) and acceptability (intention to use) are presented involving $$N=343$$ test persons altogether. The first formative evaluation study ( $$N\!=\!210$$ ) addresses the usability of the robot's article search system. It is demonstrated how several usability problems could be identified and eliminated, leading to significantly more successful article searches. The second formative evaluation study ( $$N\!=\!39$$ ) addresses the robot's adaptation to its specific task and role in the home improvement store. Embodiment, mobility, voice output, and social behavior were analyzed and adapted to user requirements. The third summative evaluation study ( $$N=94$$ ) experimentally tested robot-assisted shopping against conventional shopping regarding core usability criteria (effectiveness, efficiency, and satisfaction). It reveals that robot-assisted shopping was as effective and satisfactory as conventional shopping. Still, at the current state of technology shopping with robot assistance was slightly slower and therefore less efficient. For all three studies the effects of users' gender, age, educational background and computer skills levels on robot usability and acceptability are presented and discussed. [ABSTRACT FROM AUTHOR]

The paper describes a multi-modal scheme for human–robot interaction suited for a wide range of intelligent service robot applications. Operating in un-engineered, cluttered, and crowded environments, such robots have to be able to actively contact potential users in their surroundings and to offer their services in an appropriate manner. Starting from a real application scenario, the usage of a robot as mobile information kiosk in a home store, some reliable methods for vision-based interaction, sound analysis and speech output have been developed. These methods are integrated into a prototypical interaction cycle that can be assumed as a general approach to human–machine interaction. Experimental results demonstrate the strengths and weaknesses of the proposed methods. [Copyright &y& Elsevier]

Robustly estimating the orientations of people is a crucial precondition for a wide range of applications. Especially for autonomous systems operating in populated environments, the orientation of a person can give valuable information to increase their acceptance. Given people's orientations, mobile systems can apply navigation strategies which take people's proxemics into account or approach them in a human like manner to perform human robot interaction (HRI) tasks. In this paper, we present an approach for person orientation estimation based on computationally efficient features extracted from colored point clouds, formerly used for a two-class person attribute classification. The classification approach has been extended to the continuous domain while treating the problem of orientation estimation in real time. Furthermore, we present an approach for tracking estimated orientations over time using a Bayesian filter. We will show that tracking can increase the accuracy of orientations by up to 3. 69 ° on a dataset recorded with a mobile robot. Best results on this highly challenging dataset are achieved with a regression approach for orientation estimation in combination with tracking. The mean angular error of just 16. 49 ° proofs the applicability in real-world scenarios. • Real-time orientation estimation and tracking approach on CPU only. • Extensive evaluation on a real-world dataset. • Tracking improves accuracy and robustness. • A mean orientation accuracy of 16.49 degree is achieved. [ABSTRACT FROM AUTHOR]