Search

This paper introduces an innovative approach to 3D environmental mapping through the integration of a compact, handheld sensor package with a two-stage sensor fusion pipeline. The sensor package, incorporating LiDAR, IMU, RGB, and thermal cameras, enables comprehensive and robust 3D mapping of various environments. By leveraging Simultaneous Localization and Mapping (SLAM) and thermal imaging, our solution offers good performance in conditions where global positioning is unavailable and in visually degraded environments. The sensor package runs a real-time LiDAR-Inertial SLAM algorithm, generating a dense point cloud map that accurately reconstructs the geometric features of the environment. Following the acquisition of that point cloud, we post-process these data by fusing them with images from the RGB and thermal cameras and produce a detailed, color-enriched 3D map that is useful and adaptable to different mission requirements. We demonstrated our system in a variety of scenarios, from indoor to outdoor conditions, and the results showcased the effectiveness and applicability of our sensor package and fusion pipeline. This system can be applied in a wide range of applications, ranging from autonomous navigation to smart agriculture, and has the potential to make a substantial benefit across diverse fields. [ABSTRACT FROM AUTHOR]

LiDAR sensors, pivotal in various fields like agriculture and robotics for tasks such as 3D object detection and map creation, are increasingly coupled with thermal cameras to harness heat information. This combination proves particularly effective in adverse conditions like darkness and rain. Ensuring seamless fusion between the sensors necessitates precise extrinsic calibration. Our innovative calibration method leverages human presence during sensor setup movements, eliminating the reliance on dedicated calibration targets. It optimizes extrinsic parameters by employing a novel evolutionary algorithm on a specifically designed loss function that measures human alignment across modalities. Our approach showcases a notable 4.43 % improvement in the loss over extrinsic parameters obtained from target-based calibration in the FieldSAFE dataset. This advancement reduces costs related to target creation, saves time in diverse pose collection, mitigates repetitive calibration efforts amid sensor drift or setting changes, and broadens accessibility by obviating the need for specific targets. The adaptability of our method in various environments, like urban streets or expansive farm fields, stems from leveraging the ubiquitous presence of humans. Our method presents an efficient, cost-effective, and readily applicable means of extrinsic calibration, enhancing sensor fusion capabilities in the critical fields reliant on precise and robust data acquisition. [ABSTRACT FROM AUTHOR]

Self-driving vehicles demand efficient and reliable depth-sensing technologies. Lidar, with its capability for long-distance, high-precision measurement, is a crucial component in this pursuit. However, conventional mechanical scanning implementations suffer from reliability, cost, and frame rate limitations. Solid-state lidar solutions have emerged as a promising alternative, but the vast amount of photon data processed and stored using conventional direct time-of-flight (dToF) prevents long-distance sensing unless power-intensive partial histogram approaches are used. In this paper, we introduce a groundbreaking 'guided' dToF approach, harnessing external guidance from other onboard sensors to narrow down the depth search space for a power and data-efficient solution. This approach centers around a dToF sensor in which the exposed time window of independent pixels can be dynamically adjusted. We utilize a 64-by-32 macropixel dToF sensor and a pair of vision cameras to provide the guiding depth estimates. Our demonstrator captures a dynamic outdoor scene at 3 fps with distances up to 75 m. Compared to a conventional full histogram approach, on-chip data is reduced by over twenty times, while the total laser cycles in each frame are reduced by at least six times compared to any partial histogram approach. The capability of guided dToF to mitigate multipath reflections is also demonstrated. For self-driving vehicles where a wealth of sensor data is already available, guided dToF opens new possibilities for efficient solid-state lidar. [ABSTRACT FROM AUTHOR]

Over the last decade, robotic perception algorithms have significantly benefited from the rapid advances in deep learning (DL). Indeed, a significant amount of the autonomy stack of different commercial and research platforms relies on DL for situational awareness, especially vision sensors. This work explored the potential of general-purpose DL perception algorithms, specifically detection and segmentation neural networks, for processing image-like outputs of advanced lidar sensors. Rather than processing the three-dimensional point cloud data, this is, to the best of our knowledge, the first work to focus on low-resolution images with a 360° field of view obtained with lidar sensors by encoding either depth, reflectivity, or near-infrared light in the image pixels. We showed that with adequate preprocessing, general-purpose DL models can process these images, opening the door to their usage in environmental conditions where vision sensors present inherent limitations. We provided both a qualitative and quantitative analysis of the performance of a variety of neural network architectures. We believe that using DL models built for visual cameras offers significant advantages due to their much wider availability and maturity compared to point cloud-based perception. [ABSTRACT FROM AUTHOR]

Recently, 3D object detection based on multi-modal sensor fusion has been increasingly adopted in automated driving and robotics. For example, the semantic information provided by cameras and the geometric information provided by light detection and ranging (LiDAR) are fused to perceive 3D objects, as single modal sensors are unable to capture enough information from the environment. Many state-of-the-art methods fuse the signals sequentially for simplicity. By sequentially, we mean using the image semantic signals as auxiliary input for LiDAR-based object detectors would make the overall performance heavily rely on the semantic signals. Moreover, the error introduced by these signals may lead to detection errors. To remedy this dilemma, we propose an approach coined supervised-PointRendering to correct the potential errors in the image semantic segmentation results by training auxiliary tasks with fused features of the laser point geometry feature, the image semantic feature and a novel laser visibility feature. The laser visibility feature is obtained through the raycasting algorithm and is adopted to constrain the spatial distribution of fore- and background objects. Furthermore, we build an efficient anchor-free Single Stage Detector (SSD) powered by an advanced global-optimal label assignment to achieve a better time–accuracy balance. The new detection framework is evaluated on the extensively used KITTI and nuScenes datasets, manifesting the highest inference speed and at the same time outperforming most of the existing single-stage detectors with respect to the average precision. [ABSTRACT FROM AUTHOR]

The safety of vehicles is one of the major goals of driving automation. The safety distance is longer for rail vehicles such as trams because of the adherence limitations of the wheel-to-rail system. The major issues of fixed frontal sensing are fake target detection, blind spots related to rail slopes, curves, and random changes in the target's illumination or reflectivity. In this experimental study, distance measurements were performed using a scaled tram model equipped with a LiDAR sensor with a narrow field of view, under different conditions of illumination, size, and reflectivity of the target objects, and using different track configurations, to evaluate the effectiveness of such sensors in collision-avoidance systems for rail applications. The experimental findings are underlining the sensor's sensitivity to fake targets, objects in the sensor's blind spots, and special optical interferences, which are important for evaluating long-range LiDAR capabilities in sensing safety distance for vehicles. The conclusions can help developers to produce a dedicated colliding prevention system for trams and to identify the zones with high risk in the track where additional protection methods should be used. The LiDAR sensor must be used in conjunction with additional sensors to perform all the security tasks of an anti-colliding system for the tram. [ABSTRACT FROM AUTHOR]

Foot-mounted inertial sensors can be used for indoor pedestrian navigation. In this paper, to improve the accuracy of pedestrian location, we propose a method using a distance sensor (LIDAR) in addition to an inertial measurement unit (IMU). The distance sensor is a time of flight range finder with 30 m measurement range (at 33.33 Hz). Using a distance sensor, walls on corridors are automatically detected. The detected walls are used to correct the heading of the pedestrian path. Through experiments, it is shown that the accuracy of the heading is significantly improved using the proposed algorithm. Furthermore, the system is shown to work robustly in indoor environments with many doors and passing people. [ABSTRACT FROM AUTHOR]

We study the nonlinear dynamics of a quantum cascade laser (QCL) with a strong reinjection provided by the feedback from two external targets in a double cavity configuration. The nonlinear coupling of interferometric signals from the two targets allows us to propose a displacement sensor with nanometric resolution. The system exploits the ultra-stability of QCLs in self-mixing configuration to access the intrinsic nonlinearity of the laser, described by the Lang-Kobayashi model, and it relies on a stroboscopic-like effect in the voltage signal registered at the QCL terminals that relates the "slow" target motion to the "fast" target one. [ABSTRACT FROM AUTHOR]

An automated laser rangefinding instrument was developed to characterize overstorey and understorey vegetation dynamics over time. Design criteria were based on information needs within the statewide forest monitoring program in Victoria, Australia. The ground-based monitoring instrument captures the key vegetation structural information needed to overcome ambiguity in the estimation of forest Leaf Area Index (LAI) from satellite sensors. The scanning lidar instrument was developed primarily from low cost, commercially accessible components. While the 635 nm wavelength lidar is not ideally suited to vegetation studies, there was an acceptable trade-off between cost and performance. Tests demonstrated reliable range estimates to live foliage up to a distance of 60 m during night-time operation. Given the instrument's scan angle of 57.5 degrees zenith, the instrument is an effective tool for monitoring LAI in forest canopies up to a height of 30 m. An 18 month field trial of three co-located instruments showed consistent seasonal trends and mean LAI of between 1.32 to 1.56 and a temporal LAI variation of 8 to 17% relative to the mean. [ABSTRACT FROM AUTHOR]

Mapping the environment is necessary for navigation, planning and manipulation. In this paper, a fusion framework (as data-in-decision-out) is introduced for a 2D LIDAR and a 3D ultrasonic sensor to achieve three-dimensional mapping without expensive 3D LiDAR scanner or visual processing. Two sensor models are proposed for the two sensors used for map updating. Furthermore, 2D/3D map representations are discussed for our fusion approach. We also compare different probabilistic fusion methods and discuss criterias for choosing appropriate methods. Experiments are carried out with a real ground robot platform in an indoor environment. The 2D and 3D map results demonstrate that our approach is able to show the surrounding in more details. Sensor fusion provides a better estimation of the environment and the ego-pose whilst lowering the necessary resources. This gives the robot's perception of the environment more information by using only one additional low-cost 3D ultrasonic sensor. This is especially important for robust and light-weight robots with limited resources. [ABSTRACT FROM AUTHOR]