Search

One of the challenging problems in robot navigation is efficient and safe planning in a highly dynamic environment, where the robot is required to understand pedestrian patterns in the environment, such as train station. The rapid movement of pedestrians makes the robot more difficult to solve the collision problem. In this paper, we propose a navigation probability map to solve the pedestrians&rsquo, rapid movement problem based on the influencer recognition model (IRM). The influencer recognition model (IRM) is a data-driven model to infer a distribution over possible causes of pedestrian&rsquo, s turning. With this model, we can obtain a navigation probability map by analyzing the changes in the effective pedestrian trajectory. Finally, we combined navigation probability map and artificial potential field (APF) method to propose a robot navigation method and verified it on our data-set, which is an unobstructed, overlooked pedestrians&rsquo, data-set collected by us.

Autonomous Vehicles (AVs) have the potential to solve many traffic problems, such as accidents, congestion and pollution. However, there are still challenges to overcome, for instance, AVs need to accurately perceive their environment to safely navigate in busy urban scenarios. The aim of this paper is to review recent articles on computer vision techniques that can be used to build an AV perception system. AV perception systems need to accurately detect non-static objects and predict their behaviour, as well as to detect static objects and recognise the information they are providing. This paper, in particular, focuses on the computer vision techniques used to detect pedestrians and vehicles. There have been many papers and reviews on pedestrians and vehicles detection so far. However, most of the past papers only reviewed pedestrian or vehicle detection separately. This review aims to present an overview of the AV systems in general, and then review and investigate several detection computer vision techniques for pedestrians and vehicles. The review concludes that both traditional and Deep Learning (DL) techniques have been used for pedestrian and vehicle detection; however, DL techniques have shown the best results. Although good detection results have been achieved for pedestrians and vehicles, the current algorithms still struggle to detect small, occluded, and truncated objects. In addition, there is limited research on how to improve detection performance in difficult light and weather conditions. Most of the algorithms have been tested on well-recognised datasets such as Caltech and KITTI; however, these datasets have their own limitations. Therefore, this paper recommends that future works should be implemented on more new challenging datasets, such as PIE and BDD100K. EPSRC DTP PhD studentship

Various remote sensing technologies have been applied in intelligent vehicles and robots for surrounding-environment recognition. However, these technologies experience difficulties in detecting pedestrians in blind areas and their motions, such as rush-out behaviors. To address this issue, we present a radar-based technique for the detection of pedestrians in blind areas and the classification of different risks of rush-out behaviors among detected pedestrians. We verify their ability to detect pedestrian motion in blind areas by conducting experiments in two environments with blind areas formed by outdoor cars and indoor walls. Then, the classification of motions with different risks of rush-out behaviors among pedestrians detected in the blind areas is demonstrated. We use the clustering method to accurately classify several types of behaviors with different rush-out risks in both environments.

Regular washing of public pavements is necessary to ensure that the public environment is sanitary for social activities. This is a challenge for autonomous cleaning robots, as they must adapt to the environment with varying pavement widths while avoiding pedestrians. A self-reconfigurable pavement sweeping robot, named Panthera, has the mechanisms to perform reconfiguration in width to enable smooth cleaning operations, and it changes its behavior based on environment dynamics of moving pedestrians and changing pavement widths. Reconfiguration in the robot’s width is possible, due to the scissor mechanism at the core of the robot’s body, which is driven by a lead screw motor. Panthera will perform locomotion and reconfiguration based on perception sensors feedback control proposed while using an Red Green Blue-D (RGB-D) camera. The proposed control scheme involves publishing robot kinematic parameters for reconfiguration during locomotion. Experiments were conducted in outdoor pavements to demonstrate the autonomous reconfiguration during locomotion to avoid pedestrians while complying with varying pavements widths in a real-world scenario.

Improving road safety through artificial intelligence is now crucial to achieving more secure smart cities. With this objective, a mobile app based on the integration of the smartphone sensors and a fuzzy logic strategy to determine the pedestrian&rsquo, s crossing intention around crosswalks is presented. The app developed also allows the calculation, tracing and guidance of safe routes thanks to an optimization algorithm that includes pedestrian areas on the paths generated over the whole city through a cloud database (i.e., zebra crossings, pedestrian streets and walkways). The experimentation carried out consisted in testing the fuzzy logic strategy with a total of 31 volunteers crossing and walking around a crosswalk. For that, the fuzzy logic approach was subjected to a total of 3120 samples generated by the volunteers. It has been proven that a smartphone can be successfully used as a crossing intention detector system with an accuracy of 98.63%, obtaining a true positive rate of 98.27% and a specificity of 99.39% according to a receiver operating characteristic analysis. Finally, a total of 30 routes were calculated by the proposed algorithm and compared with Google Maps considering the values of time, distance and safety along the routes. As a result, the routes generated by the proposed algorithm were safer than the routes obtained with Google Maps, achieving an increase in the use of safe pedestrian areas of at least 183%.

Detection of traversable areas is essential to navigation of autonomous personal mobility systems in unknown pedestrian environments. However, traffic rules may recommend or require driving in specified areas, such as sidewalks, in environments where roadways and sidewalks coexist. Therefore, it is necessary for such autonomous mobility systems to estimate the areas that are mechanically traversable and recommended by traffic rules and to navigate based on this estimation. In this paper, we propose a method for weakly-supervised recommended traversable area segmentation in environments with no edges using automatically labeled images based on paths selected by humans. This approach is based on the idea that a human-selected driving path more accurately reflects both mechanical traversability and human understanding of traffic rules and visual information. In addition, we propose a data augmentation method and a loss weighting method for detecting the appropriate recommended traversable area from a single human-selected path. Evaluation of the results showed that the proposed learning methods are effective for recommended traversable area detection and found that weakly-supervised semantic segmentation using human-selected path information is useful for recommended area detection in environments with no edges.

In order to evaluate the effectiveness of a pedestrian-automatic emergency braking (PAEB) system on pedestrian protection, a set of PAEB test equipment was developed according to the test requirement of China-New Car Assessment Program (C-NCAP) (2018) in this study. In the aspect of system control strategy, global positioning system (GPS) differential positioning was used to achieve the required measurement and positioning accuracy, the collaborative control between the PAEB test equipment and automated driving robot (ADR) was achieved by wireless communication, and the motion state of the dummy target in the PAEB system was controlled by using the S-shaped-curve velocity control method. Part of the simulations and field tests were conducted according to the scenario requirements specified in C-NCAP (2018). The experimental and simulated results showed that the test equipment demonstrated high accuracy and precision in the process of testing, the dummy target movement was smooth and stable, complying with the requirements of PAEB tests set forth in C-NCAP (2018), and yielding satisfactory results as designed. Subsequently, the performance of the AEB of a vehicle under test (VUT) was conducted and the score for star-rating to evaluate the performance level of AEB calculated. Results indicated the developed test equipment in this study could be used to evaluate the performance of the PAEB system with effectiveness.

Cell-based crowd evacuation systems provide adaptive or static exit-choice indications that favor a coordinated group dynamic, improving evacuation time and safety. While a great effort has been made to modeling its control logic by assuming an ideal communication and positioning infrastructure, the architectural dimension and the influence of pedestrian positioning uncertainty have been largely overlooked. In our previous research, a cell-based crowd evacuation system (CellEVAC) was proposed that dynamically allocates exit gates to pedestrians in a cell-based pedestrian positioning infrastructure. This system provides optimal exit-choice indications through color-based indications and a control logic module built upon an optimized discrete-choice model. Here, we investigate how location-aware technologies and wearable devices can be used for a realistic deployment of CellEVAC. We consider a simulated real evacuation scenario (Madrid Arena) and propose a system architecture for CellEVAC that includes: a controller node, a radio-controlled light-emitting diode (LED) wristband subsystem, and a cell-node network equipped with active Radio Frequency Identification (RFID) devices. These subsystems coordinate to provide control, display, and positioning capabilities. We quantitatively study the sensitivity of evacuation time and safety to uncertainty in the positioning system. Results showed that CellEVAC was operational within a limited range of positioning uncertainty. Further analyses revealed that reprogramming the control logic module through a simulation optimization process, simulating the positioning system's expected uncertainty level, improved the CellEVAC performance in scenarios with poor positioning systems., Ministerio de Economía, Industria y Competitividad

P2V (pedestrian-to-vehicle) communication, in which a pedestrian&rsquo, s mobile device notifies its position to nearby vehicles in order to prevent pedestrian accidents, has attracted much research interest recently, but its performance largely depends on the precision of pedestrians&rsquo, positioning. Pedestrian positioning is generally performed by using GPS (Global Positioning System), and its precision may greatly degrade in urban canyons. To improve positioning precision of pedestrians, it was proposed that vehicles around pedestrians be used as anchors beside satellites. In this method, a pedestrian device overhears V2V (vehicle-to-vehicle) communication signals which carry the vehicle position and calculates a pedestrian&rsquo, s position using pedestrian&ndash, vehicle distance/angle information estimated from CSI (channel state information) of the V2V signals. However, angle estimation typically depends on the number of antennas of the pedestrian device. In this paper, we propose a new method to estimate signal radiation angle by tracking temporal change of CSI caused by vehicle movement during signal transmission and investigate its application to precise pedestrian positioning. Three-dimensional ray tracing simulations confirm that compared to the base method using eight antennas, the proposed method with a single antenna reduces average angle error from 13 degrees to 3.9 degrees, and reduces average positioning error from 2.49 m to 0.78 m.

Accurate identification of pedestrian crossing intention is of great significance to the safe and efficient driving of future fully automated vehicles in the city. This paper focuses on pedestrian intention recognition on the basis of pedestrian detection and tracking. A large number of natural crossing sequence data of pedestrians and vehicles are first collected by a laser scanner and HD camera, then 1980 effective crossing samples of pedestrians are selected. Influencing parameter sets of pedestrian crossing intention are then obtained through statistical analysis. Finally, long short-term memory network with attention mechanism (AT-LSTM) model is proposed. Compared with the support vector machine (SVM) model, results show that when the pedestrian crossing intention is recognized 0 s prior to crossing, the recognition accuracy of the AT-LSTM model for pedestrian crossing intention is 96.15%, which is 6.07% higher than that of SVM model, when the pedestrian crossing intention is recognized 0.6 s prior, the recognition accuracy of AT-LSTM model is 90.68%, which is 4.85% higher than that of the SVM model. The determination of pedestrian crossing intention parameter set and the more accurate recognition of pedestrian intention provided in this work provide a foundation for future fully automated driving vehicles.

One of the approaches for indoor positioning using smartphones is pedestrian dead reckoning. There, the user step length is estimated using empirical or biomechanical formulas. Such calculation was shown to be very sensitive to the smartphone location on the user. In addition, knowledge of the smartphone location can also help for direct step-length estimation and heading determination. In a wider point of view, smartphone location recognition is part of human activity recognition employed in many fields and applications, such as health monitoring. In this paper, we propose to use deep learning approaches to classify the smartphone location on the user, while walking, and require robustness in terms of the ability to cope with recordings that differ (in sampling rate, user dynamics, sensor type, and more) from those available in the train dataset. The contributions of the paper are: (1) Definition of the smartphone location recognition framework using accelerometers, gyroscopes, and deep learning, (2) examine the proposed approach on 107 people and 31 h of recorded data obtained from eight different datasets, and (3) enhanced algorithms for using only accelerometers for the classification process. The experimental results show that the smartphone location can be classified with high accuracy using only the smartphone&rsquo, s accelerometers.

Pedestrian trajectory prediction is one of the main concerns of computer vision problems in the automotive industry, especially in the field of advanced driver assistance systems. The ability to anticipate the next movements of pedestrians on the street is a key task in many areas, e.g., self-driving auto vehicles, mobile robots or advanced surveillance systems, and they still represent a technological challenge. The performance of state-of-the-art pedestrian trajectory prediction methods currently benefits from the advancements in sensors and associated signal processing technologies. The current paper reviews the most recent deep learning-based solutions for the problem of pedestrian trajectory prediction along with employed sensors and afferent processing methodologies, and it performs an overview of the available datasets, performance metrics used in the evaluation process, and practical applications. Finally, the current work exposes the research gaps from the literature and outlines potential new research directions.

As the technological advancement in the automotive field increases and the complexity of vehicle and infrastructure applications is extremely high, new directions and approaches are needed in this field. Supporting and developing vehicular applications dedicated to road safety by analyzing the current behavior of existing networks in various forms is imperative. This paper studies and implements a DSRC-type communications infrastructure that receives a set of controllable and adjustable indicators, which can provide messages to network drivers in a timely manner. The implementation is based on the 802.11p protocol and initially addresses pedestrian infrastructure or pedestrian safety, controlled areas, and perimeters that allow intelligent communications. The design and setting of the communication parameters in the lower layer of the DSRC stack for vehicle applications are part of this work, aspects that are also relevant in the case of autonomous vehicles.

Anticipating pedestrian crossing behavior in urban scenarios is a challenging task for autonomous vehicles. Early this year, a benchmark comprising JAAD and PIE datasets have been released. In the benchmark, several state-of-the-art methods have been ranked. However, most of the ranked temporal models rely on recurrent architectures. In our case, we propose, as far as we are concerned, the first self-attention alternative, based on transformer architecture, which has had enormous success in natural language processing (NLP) and recently in computer vision. Our architecture is composed of various branches which fuse video and kinematic data. The video branch is based on two possible architectures: RubiksNet and TimeSformer. The kinematic branch is based on different configurations of transformer encoder. Several experiments have been performed mainly focusing on pre-processing input data, highlighting problems with two kinematic data sources: pose keypoints and ego-vehicle speed. Our proposed model results are comparable to PCPA, the best performing model in the benchmark reaching an F1 Score of nearly 0.78 against 0.77. Furthermore, by using only bounding box coordinates and image data, our model surpasses PCPA by a larger margin (F1=0.75 vs. F1=0.72). Our model has proven to be a valid alternative to recurrent architectures, providing advantages such as parallelization and whole sequence processing, learning relationships between samples not possible with recurrent architectures.

Extrinsic camera calibration is essential for any computer vision task in a camera network. Typically, researchers place a calibration object in the scene to calibrate all the cameras in a camera network. However, when installing cameras in the field, this approach can be costly and impractical, especially when recalibration is needed. This paper proposes a novel, accurate and fully automatic extrinsic calibration framework for camera networks with partially overlapping views. The proposed method considers the pedestrians in the observed scene as the calibration objects and analyzes the pedestrian tracks to obtain extrinsic parameters. Compared to the state of the art, the new method is fully automatic and robust in various environments. Our method detect human poses in the camera images and then models walking persons as vertical sticks. We apply a brute-force method to determines the correspondence between persons in multiple camera images. This information along with 3D estimated locations of the top and the bottom of the pedestrians are then used to compute the extrinsic calibration matrices. We also propose a novel method to calibrate the camera network by only using the top and centerline of the person when the bottom of the person is not available in heavily occluded scenes. We verified the robustness of the method in different camera setups and for both single and multiple walking people. The results show that the triangulation error of a few centimeters can be obtained. Typically, it requires less than one minute of observing the walking people to reach this accuracy in controlled environments. It also just takes a few minutes to collect enough data for the calibration in uncontrolled environments. Our proposed method can perform well in various situations such as multi-person, occlusions, or even at real intersections on the street.

The AEB-P (Autonomous Emergency Braking Pedestrian) system has the functional requirements of avoiding the pedestrian collision and ensuring the pedestrian&rsquo, s life safety. By studying relevant theoretical systems, such as TTC (time to collision) and braking safety distance, an AEB-P warning model was established, and the traffic safety level and work area of the AEB-P warning system were defined. The upper-layer fuzzy neural network controller of the AEB-P system was designed, and the BP (backpropagation) neural network was trained by collected pedestrian longitudinal anti-collision braking operation data of experienced drivers. Also, the fuzzy neural network model was optimized by introducing the genetic algorithm. The lower-layer controller of the AEB-P system was designed based on the PID (proportional integral derivative controller) theory, which realizes the conversion of the expected speed reduction to the pressure of a vehicle braking pipeline. The relevant pedestrian test scenarios were set up based on the C-NCAP (China-new car assessment program) test standards. The CarSim and Simulink co-simulation model of the AEB-P system was established, and a multi-condition simulation analysis was performed. The results showed that the proposed control strategy was credible and reliable and could flexibly allocate early warning and braking time according to the change in actual working conditions, to reduce the occurrence of pedestrian collision accidents.

This paper presents a system based on pedestrian dead reckoning (PDR) for localization of networked mobile users, which relies only on sensors embedded in the devices and device- to-device connectivity. The user trajectory is reconstructed by measuring step by step the user displacements. Though step length can be estimated rather accurately, heading evaluation is extremely problematic in indoor environments. Magnetometer is typically used, however measurements are strongly perturbed. To improve the location accuracy, this paper proposes a novel cooperative system to estimate the direction of motion based on a machine learning approach for perturbation detection and filtering, combined with a consensus algorithm for performance augmentation by cooperative data fusion at multiple devices. A first algorithm filters out perturbed magnetometer measurements based on a-priori information on the Earth&rsquo, s magnetic field. A second algorithm aggregates groups of users walking in the same direction, while a third one combines the measurements of the aggregated users in a distributed way to extract a more accurate heading estimate. To the best of our knowledge, this is the first approach that combines machine learning with consensus algorithms for cooperative PDR. Compared to other methods in the literature, the method has the advantage of being infrastructure-free, fully distributed and robust to sensor failures thanks to the pre-filtering of perturbed measurements. Extensive indoor experiments show that the heading error is highly reduced by the proposed approach thus leading to noticeable enhancements in localization performance.

The performance of indoor localization methods is highly dependent on the situations in which they are used. Various competitions on indoor localization have been held for fairly comparing the existing indoor localization methods in shared and controlled testing environments. However, it is difficult to evaluate the practical performance in industrial scenarios through the existing competitions. This paper introduces two indoor localization competitions, which are named the &ldquo, PDR Challenge in Warehouse Picking 2017&rdquo, and &ldquo, xDR Challenge for Warehouse Operations 2018&rdquo, for tracking workers and vehicles in a warehouse scenario. For the PDR Challenge in Warehouse Picking 2017, we conducted a unique competition based on the data measured during the actual picking operation in an actual warehouse. We term the dead-reckoning of a vehicle as vehicle dead-reckoning (VDR), and the term &ldquo, xDR&rdquo, is derived from pedestrian dead-reckoning (PDR) plus VDR. As a sequel competition of the PDR Challenge in Warehouse Picking 2017, the xDR Challenge for Warehouse Operations 2018 was conducted as the world&rsquo, s first competition that deals with tracking forklifts by VDR with smartphones. In the paper, first, we briefly summarize the existing competitions, and clarify the characteristics of our competitions by comparing them with other competitions. Our competitions have the unique capability of evaluating the practical performance in a warehouse by using the actual measured data as the test data and applying multi-faceted evaluation metrics. As a result, we successfully organize the competitions due to the many participants from many countries. As a conclusion of the paper, we summarize the findings of the competitions.

Accurate stride-length estimation is a fundamental component in numerous applications, such as pedestrian dead reckoning, gait analysis, and human activity recognition. The existing stride-length estimation algorithms work relatively well in cases of walking a straight line at normal speed, but their error overgrows in complex scenes. Inaccurate walking-distance estimation leads to huge accumulative positioning errors of pedestrian dead reckoning. This paper proposes TapeLine, an adaptive stride-length estimation algorithm that automatically estimates a pedestrian&rsquo, s stride-length and walking-distance using the low-cost inertial-sensor embedded in a smartphone. TapeLine consists of a Long Short-Term Memory module and Denoising Autoencoders that aim to sanitize the noise in raw inertial-sensor data. In addition to accelerometer and gyroscope readings during stride interval, extracted higher-level features based on excellent early studies were also fed to proposed network model for stride-length estimation. To train the model and evaluate its performance, we designed a platform to collect inertial-sensor measurements from a smartphone as training data, pedestrian step events, actual stride-length, and cumulative walking-distance from a foot-mounted inertial navigation system module as training labels at the same time. We conducted elaborate experiments to verify the performance of the proposed algorithm and compared it with the state-of-the-art SLE algorithms. The experimental results demonstrated that the proposed algorithm outperformed the existing methods and achieves good estimation accuracy, with a stride-length error rate of 4.63% and a walking-distance error rate of 1.43% using inertial-sensor embedded in smartphone without depending on any additional infrastructure or pre-collected database when a pedestrian is walking in both indoor and outdoor complex environments (stairs, spiral stairs, escalators and elevators) with natural motion patterns (fast walking, normal walking, slow walking, running, jumping).

Pedestrian Dead Reckoning (PDR)-based pedestrian navigation technology is an important part of indoor and outdoor seamless positioning services. To improve the performance of PDR, we have conducted research on a step length estimator. Firstly, based on the basic theory of inertial navigation, we analyze in detail the errors in traditional Strapdown Inertial Navigation Systems (SINSs) caused by the unique motion state of pedestrians. Then, according to the fact that the inertial data from the foot can directly reflect the gait characteristics, we conduct a step length estimator that does not rely on SINS. The experimental results show that accuracy of the proposed method is between 0.6% and 1.4% with a standard deviation of 0.25%.

Cooperative perception, or collective perception (CP), is an emerging and promising technology for intelligent transportation systems (ITS). It enables an ITS station (ITS-S) to share its local perception information with others by means of vehicle-to-X (V2X) communication, thereby achieving improved efficiency and safety in road transportation. In this paper, we present our recent progress on the development of a connected and automated vehicle (CAV) and intelligent roadside unit (IRSU). The main contribution of the work lies in investigating and demonstrating the use of CP service within intelligent infrastructure to improve awareness of vulnerable road users (VRU) and thus safety for CAVs in various traffic scenarios. We demonstrate in experiments that a connected vehicle (CV) can &ldquo, see&rdquo, a pedestrian around the corners. More importantly, we demonstrate how CAVs can autonomously and safely interact with walking and running pedestrians, relying only on the CP information from the IRSU through vehicle-to-infrastructure (V2I) communication. This is one of the first demonstrations of urban vehicle automation using only CP information. We also address in the paper the handling of collective perception messages (CPMs) received from the IRSU, and passing them through a pipeline of CP information coordinate transformation with uncertainty, multiple road user tracking, and eventually path planning/decision-making within the CAV. The experimental results were obtained with manually driven CV, fully autonomous CAV, and an IRSU retrofitted with vision and laser sensors and a road user tracking system.

Far-infrared pedestrian detection approaches for advanced driver-assistance systems based on high-dimensional features fail to simultaneously achieve robust and real-time detection. We propose a robust and real-time pedestrian detection system characterized by novel candidate filters, novel pedestrian features and multi-frame approval matching in a coarse-to-fine fashion. Firstly, we design two filters based on the pedestrians' head and the road to select the candidates after applying a pedestrian segmentation algorithm to reduce false alarms. Secondly, we propose a novel feature encapsulating both the relationship of oriented gradient distribution and the code of oriented gradient to deal with the enormous variance in pedestrians' size and appearance. Thirdly, we introduce a multi-frame approval matching approach utilizing the spatiotemporal continuity of pedestrians to increase the detection rate. Large-scale experiments indicate that the system works in real time and the accuracy has improved about 9% compared with approaches based on high-dimensional features only.

Perception is a vital part of driving. Every year, the loss in visibility due to snow, fog, and rain causes serious accidents worldwide. Therefore, it is important to be aware of the impact of weather conditions on perception performance while driving on highways and urban traffic in all weather conditions. The goal of this paper is to provide a survey of sensing technologies used to detect the surrounding environment and obstacles during driving maneuvers in different weather conditions. Firstly, some important historical milestones are presented. Secondly, the state-of-the-art automated driving applications (adaptive cruise control, pedestrian collision avoidance, etc.) are introduced with a focus on all-weather activity. Thirdly, the most involved sensor technologies (radar, lidar, ultrasonic, camera, and far-infrared) employed by automated driving applications are studied. Furthermore, the difference between the current and expected states of performance is determined by the use of spider charts. As a result, a fusion perspective is proposed that can fill gaps and increase the robustness of the perception system.

Pedestrian detection is an essential problem of computer vision, which has achieved tremendous success under controllable conditions using visible light imaging sensors in recent years. However, most of them do not consider low-light environments which are very common in real-world applications. In this paper, we propose a novel pedestrian detection algorithm using multi-task learning to address this challenge in low-light environments. Specifically, the proposed multi-task learning method is different from the most commonly used multi-task learning method&mdash, the parameter sharing mechanism&mdash, in deep learning. We design a novel multi-task learning method with feature-level fusion and a sharing mechanism. The proposed approach contains three parts: an image relighting subnetwork, a pedestrian detection subnetwork, and a feature-level multi-task fusion learning module. The image relighting subnetwork adjusts the low-light image quality for detection, the pedestrian detection subnetwork learns enhanced features for prediction, and the feature-level multi-task fusion learning module fuses and shares features among component networks for boosting image relighting and detection performance simultaneously. Experimental results show that the proposed approach consistently and significantly improves the performance of pedestrian detection on low-light images obtained by visible light imaging sensor.

In recent years, research and development of autonomous driving technology have gained much interest. Many autonomous driving frameworks have been developed in the past. However, building a safely operating fully functional autonomous driving framework is still a challenge. Several accidents have been occurred with autonomous vehicles, including Tesla and Volvo XC90, resulting in serious personal injuries and death. One of the major reasons is the increase in urbanization and mobility demands. The autonomous vehicle is expected to increase road safety while reducing road accidents that occur due to human errors. The accurate sensing of the environment and safe driving under various scenarios must be ensured to achieve the highest level of autonomy. This research presents Clothoid, a unified framework for fully autonomous vehicles, that integrates the modules of HD mapping, localization, environmental perception, path planning, and control while considering the safety, comfort, and scalability in the real traffic environment. The proposed framework enables obstacle avoidance, pedestrian safety, object detection, road blockage avoidance, path planning for single-lane and multi-lane routes, and safe driving of vehicles throughout the journey. The performance of each module has been validated in K-City under multiple scenarios where Clothoid has been driven safely from the starting point to the goal point. The vehicle was one of the top five to successfully finish the autonomous vehicle challenge (AVC) in the Hyundai AVC.

open access article The field of cooperative intelligent transport systems and more specifically pedestrians to vehicles could be characterized as quite challenging, since there is a broad research area to be studied, with direct positive results to society. Pedestrians to vehicles is a type of cooperative intelligent transport system, within the group of early warning collision/safety system. In this article, we examine the research and applications carried out so far within the field of pedestrians to vehicles cooperative transport systems by leveraging the information coming from vulnerable road users’ smartphones. Moreover, an extensive literature review has been carried out in the fields of vulnerable road users outdoor localisation via smartphones and vulnerable road users next step/movement prediction, which are closely related to pedestrian to vehicle applications and research. We identify gaps that exist in these fields that could be improved/extended/enhanced or newly developed, while we address future research objectives and methodologies that could support the improvement/development of those identified gaps.

As a result of its important role in video surveillance, pedestrian attribute recognition has become an attractive facet of computer vision research. Because of the changes in viewpoints, illumination, resolution and occlusion, the task is very challenging. In order to resolve the issue of unsatisfactory performance of existing pedestrian attribute recognition methods resulting from ignoring the correlation between pedestrian attributes and spatial information, in this paper, the task is regarded as a spatiotemporal, sequential, multi-label image classification problem. An attention-based neural network consisting of convolutional neural networks (CNN), channel attention (CAtt) and convolutional long short-term memory (ConvLSTM) is proposed (CNN-CAtt-ConvLSTM). Firstly, the salient and correlated visual features of pedestrian attributes are extracted by pre-trained CNN and CAtt. Then, ConvLSTM is used to further extract spatial information and correlations from pedestrian attributes. Finally, pedestrian attributes are predicted with optimized sequences based on attribute image area size and importance. Extensive experiments are carried out on two common pedestrian attribute datasets, PEdesTrian Attribute (PETA) dataset and Richly Annotated Pedestrian (RAP) dataset, and higher performance than other state-of-the-art (SOTA) methods is achieved, which proves the superiority and validity of our method.

The ubiquity of sensor-rich smartphones provides opportunities for a low-cost method to track indoor pedestrians. In this situation, pedestrian dead reckoning (PDR) is a widely used technology, however, its cumulative error seriously affects its accuracy. This paper presents a method of combining infrastructure-free indoor acoustic self-positioning with PDR self-positioning, which verifies the rationality of PDR results through the acoustic constraint between a sound source and its image sources. We further determine the first-order echo delay measurements, thus obtaining the mobile user position. We verify that the proposed method can achieve a continuous self-positioning median error of 0.19 m, and the error probability below 0.12 m is 54.46%, which indicates its ability to eliminate PDR error, as well as its adaptability to environmental disturbances.

Human activity recognition and pedestrian dead reckoning are an interesting field because of their importance utilities in daily life healthcare. Currently, these fields are facing many challenges, one of which is the lack of a robust algorithm with high performance. This paper proposes a new method to implement a robust step detection and adaptive distance estimation algorithm based on the classification of five daily wrist activities during walking at various speeds using a smart band. The key idea is that the non-parametric adaptive distance estimator is performed after two activity classifiers and a robust step detector. In this study, two classifiers perform two phases of recognizing five wrist activities during walking. Then, a robust step detection algorithm, which is integrated with an adaptive threshold, peak and valley correction algorithm, is applied to the classified activities to detect the walking steps. In addition, the misclassification activities are fed back to the previous layer. Finally, three adaptive distance estimators, which are based on a non-parametric model of the average walking speed, calculate the length of each strike. The experimental results show that the average classification accuracy is about 99%, and the accuracy of the step detection is 98.7%. The error of the estimated distance is 2.2&ndash, 4.2% depending on the type of wrist activities.

Indoor positioning is currently a research hotspot. In recent years, Pedestrian Dead Reckoning (PDR) has been widely used in indoor positioning. However, the positioning error caused by heading drifts will accumulate as the walking distance increases, so some methods need to be used to correct the heading angle. Heuristic Drift Elimination (HDE) is an effective heading correction algorithm, which only uses the information of a building’s dominant directions to reduce the heading error, but it does not apply to the non-dominant direction condition. In this paper, we propose a heading drift suppressing method for the limitation of HDE. Firstly, the method constructs membership functions to judge the pedestrian’s motion according to the result of comprehensive evaluation. Then, it further determines by a threshold condition whether the pedestrian walks along the dominant directions, and a heading error measurement is introduced for heading correction. Finally, we verify by experiments that the proposed method can correct heading angles properly for different conditions, which indicates an adaptability to the environment.

Aimed at overcoming the problems of cumulative errors and low positioning accuracy in single Inertial Navigation Systems (INS), an Optimal Enhanced Kalman Filter (OEKF) is proposed in this paper to achieve accurate positioning of pedestrians within an enclosed environment. Firstly, the errors of the inertial sensors are analyzed, modeled, and reconstructed. Secondly, the cumulative errors in attitude and velocity are corrected using the attitude fusion filtering algorithm and Zero Velocity Update algorithm (ZUPT), respectively. Then, the OEKF algorithm is described in detail. Finally, a pedestrian indoor positioning experimental platform is established to verify the performance of the proposed positioning system. Experimental results show that the accuracy of the pedestrian indoor positioning system can reach 0.243 m, giving it a high practical value.

The objective of this article is to study the problem of pedestrian classification across different light spectrum domains (visible and far-infrared (FIR)) and modalities (intensity, depth and motion). In recent years, there has been a number of approaches for classifying and detecting pedestrians in both FIR and visible images, but the methods are difficult to compare, because either the datasets are not publicly available or they do not offer a comparison between the two domains. Our two primary contributions are the following: (1) we propose a public dataset, named RIFIR , containing both FIR and visible images collected in an urban environment from a moving vehicle during daytime, and (2) we compare the state-of-the-art features in a multi-modality setup: intensity, depth and flow, in far-infrared over visible domains. The experiments show that features families, intensity self-similarity (ISS), local binary patterns (LBP), local gradient patterns (LGP) and histogram of oriented gradients (HOG), computed from FIR and visible domains are highly complementary, but their relative performance varies across different modalities. In our experiments, the FIR domain has proven superior to the visible one for the task of pedestrian classification, but the overall best results are obtained by a multi-domain multi-modality multi-feature fusion.

Location-based services (LBS) have attracted a great deal of attention recently. Outdoor localization can be solved by the GPS technique, but how to accurately and efficiently localize pedestrians in indoor environments is still a challenging problem. Recent techniques based on WiFi or pedestrian dead reckoning (PDR) have several limiting problems, such as the variation of WiFi signals and the drift of PDR. An auxiliary tool for indoor localization is landmarks, which can be easily identified based on specific sensor patterns in the environment, and this will be exploited in our proposed approach. In this work, we propose a sensor fusion framework for combining WiFi, PDR and landmarks. Since the whole system is running on a smartphone, which is resource limited, we formulate the sensor fusion problem in a linear perspective, then a Kalman filter is applied instead of a particle filter, which is widely used in the literature. Furthermore, novel techniques to enhance the accuracy of individual approaches are adopted. In the experiments, an Android app is developed for real-time indoor localization and navigation. A comparison has been made between our proposed approach and individual approaches. The results show significant improvement using our proposed framework. Our proposed system can provide an average localization accuracy of 1 m. Published version

This paper investigates the robustness of a new thermal-infrared pedestrian detection system under different outdoor environmental conditions. In first place the algorithm for pedestrian ROI extraction in thermal-infrared video based on both thermal and motion information is introduced. Then, the evaluation of the proposal is detailed after describing the complete thermal and motion information fusion. In this sense, the environment chosen for evaluation is described, and the twelve test sequences are specified. For each of the sequences captured from a forward-looking infrared FLIR A-320 camera, the paper explains the weather and light conditions under which it was captured. The results allow us to draw firm conclusions about the conditions under which it can be affirmed that it is efficient to use our thermal-infrared proposal to robustly extract human ROIs.

Automobile driver information as displayed on marked road signs indicates the state of the road, traffic conditions, proximity to schools, etc. These signs are important to insure the safety of the driver and pedestrians. They are also important input to the automated advanced driver assistance system (ADAS), installed in many automobiles. Over time, the arrow-road markings may be eroded or otherwise damaged by automobile contact, making it difficult for the driver to correctly identify the marking. Failure to properly identify an arrow-road marker creates a dangerous situation that may result in traffic accidents or pedestrian injury. Very little research exists that studies the problem of automated identification of damaged arrow-road marking painted on the road. In this study, we propose a method that uses a convolutional neural network (CNN) to recognize six types of arrow-road markings, possibly damaged, by visible light camera sensor. Experimental results with six databases of Road marking dataset, KITTI dataset, Málaga dataset 2009, Málaga urban dataset, Naver street view dataset, and Road/Lane detection evaluation 2013 dataset, show that our method outperforms conventional methods.

The interior space of large-scale buildings, such as hospitals, with a variety of departments, is so complicated that people may easily lose their way while visiting. Difficulties in wayfinding can cause stress, anxiety, frustration and safety issues to patients and families. An indoor navigation system including route planning and localization is utilized to guide people from one place to another. The localization of moving subjects is a critical-function component in an indoor navigation system. Pedestrian dead reckoning (PDR) is a technology that is widely employed for localization due to the advantage of being independent of infrastructure. To improve the accuracy of the localization system, combining different technologies is one of the solutions. In this study, a multi-sensor fusion approach is proposed to improve the accuracy of the PDR system by utilizing a light sensor, Bluetooth and map information. These simple mechanisms are applied to deal with the issue of accumulative error by identifying edge and sub-edge information from both Bluetooth and the light sensor. Overall, the accumulative error of the proposed multi-sensor fusion approach is below 65 cm in different cases of light arrangement. Compared to inertial sensor-based PDR system, the proposed multi-sensor fusion approach can improve 90% of the localization accuracy in an environment with an appropriate density of ceiling-mounted lamps. The results demonstrate that the proposed approach can improve the localization accuracy by utilizing multi-sensor data and fulfill the feasibility requirements of localization in an indoor navigation system.

Indoor pedestrian tracking has been identified as a key technology for indoor location-based services such as emergency locating, advertising, and gaming. However, existing smartphone-based approaches to pedestrian tracking in indoor environments have various limitations including a high cost of infrastructure constructing, labor-intensive fingerprint collection, and a vulnerability to moving obstacles. Moreover, our empirical study reveals that the accuracy of indoor locations estimated by a smartphone Inertial Measurement Unit (IMU) decreases severely when the pedestrian is arbitrarily wandering with an unstable speed. To improve the indoor tracking performance by enhancing the location estimation accuracy, we exploit smartphone-based acoustic techniques and propose an infrastructure-free indoor pedestrian tracking approach, called iIPT. The novelty of iIPT lies in the pedestrian speed reliability metric, which characterizes the reliability of the pedestrian speed provided by the smartphone IMU, and in a speed enhancing method, where we adjust a relatively less reliable pedestrian speed to the more reliable speed of a passing by &ldquo, enhancer&rdquo, based on the acoustic Doppler effect. iIPT thus changes the encountered pedestrians from an&ldquo, obstacle&rdquo, into an &ldquo, enhancer.&rdquo, Extensive real-world experiments in indoor scenarios have been conducted to verify the feasibility of realizing the acoustic Doppler effect between smartphones and to identify the applicable acoustic frequency range and transmission distance while reducing battery consumption. The experiment results demonstrate that iIPT can largely improve the tracking accuracy and decrease the average error compared with a conventional IMU-based method.

Those in the automotive industry and many researchers have become interested in the development of pedestrian protection systems in recent years. In particular, vision-based methods for predicting pedestrian intentions are now being actively studied to improve the performance of pedestrian protection systems. In this paper, we propose a vision-based system that can detect pedestrians using an on-dash camera in the car, and can then analyze their movements to determine the probability of collision. Information about pedestrians, including position, distance, movement direction, and magnitude are extracted using computer vision technologies and, using this information, a fuzzy rule-based system makes a judgement on the pedestrian&rsquo, s risk level. To verify the function of the proposed system, we built several test datasets, collected by ourselves, in high-density regions where vehicles and pedestrians mix closely. The true positive rate of the experimental results was about 86%, which shows the validity of the proposed system.