Search

Pedestrian tracking and detection have become critical aspects of advanced driver assistance systems (ADASs), due to their academic and commercial potential. Their objective is to locate various pedestrians in videos and assign them unique identities. The data association task is problematic, particularly when dealing with inter-pedestrian occlusion. This occurs when multiple pedestrians cross paths or move too close together, making it difficult for the system to identify and track individual pedestrians. Inaccurate tracking can lead to false alarms, missed detections, and incorrect decisions. To overcome this challenge, our paper focuses on improving data association in our pedestrian detection system’s Deep-SORT tracking algorithm, which is solved as a linear optimization problem using a newly generated cost matrix. We introduce a set of new data association cost matrices that rely on metrics such as intersections, distances, and bounding boxes. To evaluate trackers in real time, we use YOLOv5 to identify pedestrians in images. We also perform experimental evaluations on the Multiple Object Tracking 17 (MOT17) challenge dataset. The proposed cost matrices demonstrate promising results, showing an improvement in most MOT performance metrics compared to the default intersection over union (IOU) data association cost matrix.

Videos are full of dynamic changes along both the spatial and temporal dimensions. Large, jerky short-term motions make it difficult to extract significant changes from videos such as subtle color changes and long-term motions occurring in time-lapse sequences. In this paper, we introduce two singular value decomposition (SVD)-based video decomposition schemes to clearly reveal such changes. The first scheme involves enhancing the visual characteristics of small subtle color changes in the presence of a wide variety of motion patterns by magnifying their pixel intensities. The second scheme removes short-term motions that visually distract attention from the underlying content of video sequences such as time-lapse videos, snowing scene, and maritime surveillance. Both schemes involve the decomposition of videos into spatiotemporal slices in which each slice is further decomposed into several singular components. The low-rank components that primarily represent background and color intensity information are then temporally processed to magnify the magnitude of the signal at the subtle color change target frequency. At the same time, an approach similar to that used in denoising time-lapse sequences is applied to temporally filter the singular components representing sparse information, thereby removing jittery short-term motions while preserving long-term motions, which are represented by both low-rank and unfiltered sparse components. We demonstrate promising color magnification and motion denoising results that can be obtained much faster than results estimated using state-of-the-art techniques.

In this paper, we propose a method that removes raindrops with light field image using image inpainting. We first use the depth map generated from light field image to detect raindrop regions which are then expressed as a binary mask. The original image with raindrops is improved by refocusing on the far regions and filtering by a high-pass filter. With the binary mask and the enhanced image, image inpainting is then utilized to eliminate raindrops from the original image. We compare pre-trained models of several deep learning based image inpainting methods. A light field raindrop dataset is released to verify our method. Image quality analysis is performed to evaluate the proposed image restoration method. The recovered images are further applied to object detection and visual localization tasks.

Classifying traffic signs is an indispensable task for autonomous driving systems. Depending on the country, traffic signs possess a wide variability in their visual appearance making it harder for classification systems to succeed. Either the classifier should be fine-tuned or a bigger collection of images should be used. In this paper, we introduce a real-world European dataset for traffic sign classification. The dataset is composed of traffic sings from six European countries: Belgium, Croatia, France, Germany, The Netherlands, and Sweden. It gathers publically available datasets and complements French traffic signs with images acquired in Belfort with the equipped university autonomous vehicle. It is composed of more than 80000 images divided in 164 classes that at the same time belong to four main categories following the Vienna Convention of Road Signs. We analyzed the intra variability of classes and compared the classification performance of five convolutional neural network architectures.

In this paper, we present a novel strategy for roof segmentation from aerial images (orthophotoplans) based on the cooperation of edge- and region-based segmentation methods. The proposed strategy is composed of three major steps. The first one, called the pre-processing step, consists of simplifying the acquired image with an appropriate couple of invariant and gradient, optimized for the application, in order to limit illumination changes (shadows, brightness, etc.) affecting the images. The second step is composed of two main parallel treatments: on the one hand, the simplified image is segmented by watershed regions. Even if the first segmentation of this step provides good results in general, the image is often over-segmented. To alleviate this problem, an efficient region merging strategy adapted to the orthophotoplan particularities, with a 2D modeling of roof ridges technique, is applied. On the other hand, the simplified image is segmented by watershed lines. The third step consists of integrating both watershed segmentation strategies into a single cooperative segmentation scheme in order to achieve satisfactory segmentation results. Tests have been performed on orthophotoplans containing 100 roofs with varying complexity, and the results are evaluated with the VINETcriterion using ground-truth image segmentation. A comparison with five popular segmentation techniques of the literature demonstrates the effectiveness and the reliability of the proposed approach. Indeed, we obtain a good segmentation rate of 96% with the proposed method compared to 87.5% with statistical region merging (SRM), 84% with mean shift, 82% with color structure code (CSC), 80% with efficient graph-based segmentation algorithm (EGBIS) and 71% with JSEG.

Occupancy grid map is a popular tool for representing the surrounding environments of mobile robots/intelligent vehicles. Its applications can be dated back to the 1980s, when researchers utilized sonar or LiDAR to illustrate environments by occupancy grids. However, in the literature, research on vision-based occupancy grid mapping is scant. Furthermore, when moving in a real dynamic world, traditional occupancy grid mapping is required not only with the ability to detect occupied areas, but also with the capability to understand dynamic environments. The paper addresses this issue by presenting a stereo-vision-based framework to create a dynamic occupancy grid map, which is applied in an intelligent vehicle driving in an urban scenario. Besides representing the surroundings as occupancy grids, dynamic occupancy grid mapping could provide the motion information of the grids. The proposed framework consists of two components. The first is motion estimation for the moving vehicle itself and independent moving objects. The second is dynamic occupancy grid mapping, which is based on the estimated motion information and the dense disparity map. The main benefit of the proposed framework is the ability of mapping occupied areas and moving objects at the same time. This is very practical in real applications. The proposed method is evaluated using real data acquired by our intelligent vehicle platform “SeTCar” in urban environments.

Convolutional Network (ConvNet), with its strong image representation ability, has achieved significant progress in the computer vision and robotic fields. In this paper, we propose a visual localization approach based on place recognition that combines the powerful ConvNet features and localized image sequence matching. The image distance matrix is constructed based on the cosine distance of extracted ConvNet features, and then a sequence search technique is applied on this distance matrix for the final visual recognition. To speed up the computational efficiency, the locality sensitive hashing (LSH) method is applied to achieve real-time performances with minimal accuracy degradation. We present extensive experiments on four real world data sets to evaluate each of the specific challenges in visual recognition. A comprehensive performance comparison of different ConvNet layers (each defining a level of features) considering both appearance and illumination changes is conducted. Compared with the traditional approaches based on hand-crafted features and single image matching, the proposed method shows good performances even in the presence of appearance and illumination changes.

The Codebook model is one of the popular real-time models for background subtraction. In this paper, we first extend it from traditional Red-Green-Blue (RGB) color model to multispectral sequences. A self-adaptive mechanism is then designed based on the statistical information extracted from the data themselves, with which the performance has been improved, in addition to saving time and effort to search for the appropriate parameters. Furthermore, the Spectral Information Divergence is introduced to evaluate the spectral distance between the current and reference vectors, together with the Brightness and Spectral Distortion. Experiments on five multispectral sequences with different challenges have shown that the multispectral self-adaptive Codebook model is more capable of detecting moving objects than the corresponding RGB sequences. The proposed research framework opens a door for future works for applying multispectral sequences in moving object detection.

Stereo correspondence is one of the most active research areas in computer vision. It consists in identifying features in two or more stereo images that are generated by the same physical feature in the three-dimensional space. In our approach, the matching problem is first turned into an optimization task where a fitness function, representing the constraints on the solution, is to be minimized. The optimization process is then performed by means of a genetic algorithm with a new encoding scheme. Experimental results are presented to demonstrate the robustness and the reliability of the proposed approach for obstacle detection in front of a vehicle using linear stereo vision.

Visual localization is widely used in autonomous navigation system and Advanced Driver Assistance Systems (ADAS). However, visual-based localization in seasonal changing situations is one of the most challenging topics in computer vision and the intelligent vehicle community. The difficulty of this task is related to the strong appearance changes that occur in scenes due to weather or season changes. In this paper, a place recognition based visual localization method is proposed, which realizes the localization by identifying previously visited places using the sequence matching method. It operates by matching query image sequences to an image database acquired previously (video acquired during traveling period). In this method, in order to improve matching accuracy, multi-feature is constructed by combining a global GIST descriptor and local binary feature CSLBP (Center-symmetric local binary patterns) to represent image sequence. Then, similarity measurement according to Chi-square distance is used for effective sequences matching. For experimental evaluation, the relationship between image sequence length and sequences matching performance is studied. To show its effectiveness, the proposed method is tested and evaluated in four seasons outdoor environments. The results have shown improved precision–recall performance against the state-of-the-art SeqSLAM algorithm.

A critical concern of autonomous vehicles is safety. Different approaches have tried to enhance driving safety to reduce the number of fatal crashes and severe injuries. As an example, Intelligent Speed Adaptation (ISA) systems warn the driver when the vehicle exceeds the recommended speed limit. However, these systems only take into account fixed speed limits without considering factors like road geometry. In this paper, we consider road curvature with speed limits to automatically adjust vehicle’s speed with the ideal one through our proposed Dynamic Speed Adaptation (DSA) method. Furthermore, ‘curve analysis extraction’ and ‘speed limits database creation’ are also part of our contribution. An algorithm that analyzes GPS information off-line identifies high curvature segments and estimates the speed for each curve. The speed limit database contains information about the different speed limit zones for each traveled path. Our DSA senses speed limits and curves of the road using GPS information and ensures smooth speed transitions between current and ideal speeds. Through experimental simulations with different control algorithms on real and simulated datasets, we prove that our method is able to significantly reduce lateral errors on sharp curves, to respect speed limits and consequently increase safety and comfort for the passenger.

A precise GNSS (Global Navigation Satellite System) localization is vital for autonomous road vehicles, especially in cluttered or urban environments where satellites are occluded, preventing accurate positioning. We propose to fuse GPS (Global Positioning System) data with fisheye stereovision to face this problem independently to additional data, possibly outdated, unavailable, and needing correlation with reality. Our stereoscope is sky-facing with 360° × 180° fisheye cameras to observe surrounding obstacles. We propose a 3D modelling and plane extraction through following steps: stereoscope self-calibration for decalibration robustness, stereo matching considering neighbours epipolar curves to compute 3D, and robust plane fitting based on generated cartography and Hough transform. We use these 3D data with GPS raw data to estimate NLOS (Non Line Of Sight) reflected signals pseudorange delay. We exploit extracted planes to build a visibility mask for NLOS detection. A simplified 3D canyon model allows to compute reflections pseudorange delays. In the end, GPS positioning is computed considering corrected pseudoranges. With experimentations on real fixed scenes, we show generated 3D models reaching metric accuracy and improvement of horizontal GPS positioning accuracy by more than 50%. The proposed procedure is effective, and the proposed NLOS detection outperforms CN0-based methods (Carrier-to-receiver Noise density).

Dans cet article, nous présentons une stratégie de segmentation d'images aériennes pour la caractérisation de toitures. Pour ce faire, une méthode de segmentation d'orthophotoplans basée sur une technique de coopération intégrant une segmentation par régions et une segmentation par contours est proposée. Les deux techniques de segmentation sont assurées par l'algorithme de la Ligne de Partage des Eaux (LPE). L'approche proposée est composée de trois étapes : (i) Une étape de simplification qui consiste à simplifier l'image initiale avec un couple invariant/gradient approprié et optimisé pour l'application. Pour cela, une série de tests permettant de choisir, d'une part, l'invariant colorimétrique le plus approprié parmi 24 invariants et, d'autre part, le meilleur gradient parmi 14 gradients issus de la littérature est réalisée. (ii) La deuxième étape comporte deux stratégies de segmentation par LPE. L'image simplifiée est segmentée d'une part par une LPE-régions couplée à une stratégie de fusion de régions, et d'autre part, par une LPE-contours. Le processus de fusion de régions intègre des critères de fusion fondés sur des grandeurs radiométriques et géométriques adaptés aux particularités des orthophotoplans traités. Une technique de caractérisation 2D des arêtes de toitures par une analyse des segments est proposée afin de calculer l'un des critères de fusion. (iii) La troisième étape consiste à combiner les avantages de chaque méthode dans un même schéma de segmentation coopératif afin d'aboutir à un résultat de segmentation fiable. Les tests ont été effectués sur un orthophotoplan contenant 100 toitures de complexité variée et évaluées avec le critère de Vinet utilisant une segmentation de référence afin de prouver la robustesse et la fiabilité de l'approche proposée. La stratégie de coopération proposée permet d'aboutir à un taux de bonne segmentation de 96% contre 83% pour une LPE-régions seule ou 88% pour une LPE-contours seule.