Search

Following a demonstration path without observing exact location of an agent is a challenging navigation problem. Especially, considering the probabilistic transition function of the agent makes the problem hard to solve with an exact action decision, so learning-based approaches have been used to solve this task. For example, a previous method by Kumar and Gupta et al., robust path following network (RPF), is a neural-network-based method using visual memories of the demonstration. Although the RPF shows good performances on the path-following task, it does not consider the efficiency of the visual memory since it requires the entire visual memory of the demonstration. In this paper, we propose a path-following network using sparse memory of the demonstration path that can deal with various sparsity of the visual memory. For each time step, the proposed network makes soft attention on the sparse memory to control the agent. We test the proposed model on the Habitat simulator using MatterPort3D dataset with various sparsity of memory. The experimental results show that the proposed method achieves 81.9% of success rate and 73.7% of SPL on a model with 0.8 memory sparsity, and also the results of the models with other memory sparsity achieve reasonable performances compare to the baseline methods.

In this paper, we tackle the problem of visual target navigation which is a crucial task for embodied AI. Given a target observation and an initial observation, we extract keypoints using an off-the-shelf keypoint detection model. Then we find sparse keypoint correspondences and estimate the relative camera pose with the PnP algorithm to reach the viewpoint for the target observation. Compared to conventional approaches, our method is faster, and more robust to scene changes and occlusions. We collected a 3D scan dataset in a university building and the proposed method is verified using the dataset.

The goal of the optimal viewpoint path estimation is to generate a path to the optimal viewpoint location where the robot can best see the Point of Interest (POI). There are several learning-based methods to find an optimal viewpoint, but these methods are limited to a specific object POI and it is necessary to newly learn in a situation where a new POI is added, and not robust to the environment changes. In this paper, we propose an algorithm that generates a path to the optimal viewpoint by using the geometrical features of the environment in the situation where the target POI is in the field of view. This method makes it easy to add new POIs and is robust to environmental changes because it uses semantic and geometric information. We assume that the robot can make a simple estimation of the geometric characteristics of the surrounding environment by using pretrained networks or by using sensor values. We collected the Kwanjeong street dataset for testing our algorithm. In this dataset, the distance accuracy of our method to reach the optimal viewpoint of the POI achieved 81.8% and 70.9% for template matching accuracy.