Search

For driving safely and comfortably, the long-term trajectory prediction of surrounding vehicles is essential for autonomous vehicles. For handling the uncertain nature of trajectory prediction, deep-learning-based approaches have been proposed previously. An on-road vehicle must obey road geometry, i.e., it should run within the constraint of the road shape. Herein, we present a novel road-aware trajectory prediction method which leverages the use of high-definition maps with a deep learning network. We developed a data-efficient learning framework for the trajectory prediction network in the curvilinear coordinate system of the road and a lane assignment for the surrounding vehicles. Then, we proposed a novel output-constrained sequence-to-sequence trajectory prediction network to incorporate the structural constraints of the road. Our method uses these structural constraints as prior knowledge for the prediction network. It is not only used as an input to the trajectory prediction network, but is also included in the constrained loss function of the maneuver recognition network. Accordingly, the proposed method can predict a feasible and realistic intention of the driver and trajectory. Our method has been evaluated using a real traffic dataset, and the results thus obtained show that it is data-efficient and can predict reasonable trajectories at merging sections.

Lane detection and tracking in a complex road environment is one of the most important research areas in highly automated driving systems. Studies on lane detection cover a variety of difficulties, such as shadowy situations, dimmed lane painting, and obstacles that prohibit lane feature detection. There are several hard cases in which lane candidate features are not easily extracted from image frames captured by a driving vehicle. We have carefully selected typical scenarios in which the extraction of lane candidate features can be easily corrupted by road vehicles and road markers that lead to degradations in the understanding of road scenes, resulting in difficult decision making. We have introduced two main contributions to the interpretation of road scenes in dense traffic environments. First, to obtain robust road scene understanding, we have designed a novel framework combining a lane tracker method integrated with a camera and a radar forward vehicle tracker system, which is especially useful in dense traffic situations. We have introduced an image template occupancy matching method with the integrated vehicle tracker that makes it possible to avoid extracting irrelevant lane features caused by forward target vehicles and road markers. Second, we present a robust multi-lane detection by a tracking algorithm that incudes adjacent lanes as well as ego lanes. We verify a comprehensive experimental evaluation with a real dataset comprised of problematic road scenarios. Experimental result shows that the proposed method is very reliable for multi-lane detection at the presented difficult situations.

In this paper, we propose a lateral upper controller that predicts vehicle motion and generates a model-free LKAS steering angle by applying long-term memory (LSTM), one of the deep learning techniques. The apparent and distinct advantage of this LSTM model is that the relationship of nonlinear sensor data can be grasped and learned devoid of a mathematical model while using the time information. In this sense, the LKAS steering angle can be generated by predicting the movement of the vehicle in consideration of the past vehicle motion based on the sensor data. In addition, the time delay problem due to the difference of sampling time on each sensor can be simply solved by learning based on a time table which has a synchronized sampling time. The input values of the upper controller are the coefficient of the road model and the vehicle dynamic characteristics are obtained from the image processing data from the camera sensor. As for the target values, the steering angles in the next state are selected. The learning model was developed by learning the many-to-one LSTM prediction model with serial connection of LSTM and Fully-Connected (FC) Multilayer Perceptron (MLP). For implementation of the learning model, Tensorflow is employed and the data from a real test road was used. With this model, the learning is conducted and its effectiveness is shown by comparing with a LKAS lateral controller using a model-based multi rate Kalman Filter (MKF).

This paper clarifies an issue that the most commonly used ADAS sensors, monocular camera and radar, do not provide abundant information about dynamically changing road scenes. In order to make the sensor more useful for a wide range of ADAS functions, we present an approach to estimate the orientation of surrounding vehicles using deep neural network. We show the possibility that camera-based method can get more competitive, evaluating it on the KITTI Orientation Estimation Benchmark, and also verifying it on our test-driving scenarios. Although its localization performance is not perfect, our model is able to reliably predict the orientation when fine conditions are given. In addition, we further study on training models using synthetic dataset, and share the weakness of this method when comparing to LiDAR-based approach on several conditions such as fully-visible, lightly/heavily-occluded and shading/lighting circumstances.

Many studies using a laser scanner have been conducted in order to study the environment of vehicles in real time. The method to find the driving area using a two-dimensional lidar sensor is divided into a forward-looking lidar sensor and a downward-looking lidar sensor based on the installation method. A downward-looking lidar sensor looks at the ground, enabling it to recognize kerbs and ditches which are lower than the installation position of the sensor. However, a downward-looking lidar sensor requires pre-processing to find the road boundary. The existing sensor models cannot generate an occupancy grid map without support, as the driving area recognized through a downward-looking lidar sensor forms a circular sector shape from the sensor installation position to the road boundary. This paper proposes a road sensor model that is capable of modelling an occupancy grid. We also propose a method to generate an occupancy grid map more suitable for autonomous vehicles by presenting the occupancy grid map in curvilinear space. The proposed method was validated by an experiment at Hanyang University campus and the quantitative results obtained from that experiment. We also compared this method with three conventional sensor model methods. The experimental results show that our method performs better than the conventional methods do in terms of both visual qualities and metric qualities.

A new method for supervoxel segmentation based on a weighted graph in complex urban environment is proposed. The proposed algorithm is designed to minimise the cost of the shortest path in the weighted graph with consideration of characteristics of the sensor for complex urban environment. We collected point clouds in a complex urban environment to evaluate the proposed method. The performance of the proposed algorithm was evaluated using boundary recall and under-segmentation errors.

Vision-based lane-sensing systems require accurate and robust sensing performance in lane detection. Besides, there exists trade-off between the computational burden and processor cost, which should be considered for implementing the systems in passenger cars. In this paper, a stereo vision-based lane detection system considering sensor configuration aspects such as field of view (FOV), span pixels, resolution, etc is developed. An inverse perspective mapping method is formulated based on the relative correspondence between the left and right cameras so that the 3D road geometry can be reconstructed in a robust manner. The selection rule of the sensor configuration and specifications is investigated for a standard highway. Based on the selected sensor configurations, it is shown that sensing region range on the camera image coordinate can be determined for the best lane-sensing performance. The proposed system is implemented on a passenger car and its real-time sensing performance is verified experimentally.

Optimal task sequence planning using a simulated annealing technique is developed for a high speed assembly station comprised of two robots. In order to minimize the tact time, parts are assembled by dual robots on moving workpieces which are transferred by a precision conveyer. The sequence of operations by the dual robots is optimized to minimize the tact time while avoiding the geometric and dynamic interferences between the dual robots. The optimization problem is a type of traveling salesman problem, but is highly complicated because the target destinations for placing individual parts keep moving on the conveyer. The speed of the conveyer which carries the workpieces should be also optimized in order to maximize the throughput of the assembly system. The task sequence optimization problem is formulated together with the problem of determining the conveyer speed. An efficient algorithm for the sequence optimization is developed and solved by the simulated annealing.

Vision-based lane sensing systems require accurate and robust sensing performance in lane detection. Besides, there exists a trade-off between the computational burden and processor cost, which should be considered for implementing the systems in passenger cars. A stereo vision-based lane detection system is developed considering sensor configuration aspects. An inverse perspective mapping method is formulated based on the relative correspondence between the left and right cameras so that the 3-dimensional road geometry can be reconstructed in a robust manner. A new monitoring model for estimating the road geometry parameters is constructed to reduce the number of the measured signals. The selection of the sensor configuration and specifications is investigated by utilizing the characteristics of standard highways. Based on the sensor configurations, it is shown that appropriate sensing region on the camera image coordinate can be determined. The proposed system is implemented on a passenger car and verified experimentally.

A description is given of a system developed for supervisory control of a telemanipulator to deal with unique characteristics of teleoperation. A six-degree-of-freedom telemanipulator is graphically simulated on an IRIS workstation. There are two modes to control the telemanipulator: manual mode and supervisory mode. In the manual mode, the operator guides the manipulator using a 6D optical sensor ball; in the supervisory mode, the operator specifies a task using a menu and a mouse to point an object and a location. The system checks possible manipulator collisions with obstacles in the environment. If the system detects an impending collision in the supervisory mode, it uses heuristics to avoid it. The system can generate virtual obstacles that occupy space that the operator cannot observe through a camera. Virtual obstacles are updated by geometric logic operations whenever viewing conditions such as the camera position are changed. Such virtual obstacles are used for collision checking and collision avoidance. >

A concurrent design method of mechanical structure and control is developed for two-link high speed robots. An integrated design approach to achieve high speed positioning is explored, in which comprehensive design parameters describing arm link geometry, actuator locations, and feedback gains are optimized with respect to the settling time of the system. First, a two-link, nonrigid arm is analyzed and a simple dynamic model representing rapid positioning processes is obtained. Optimal feedback gains minimizing the settling time are obtained as functions of structural parameters involved in the dynamic model. The structural parameters are then optimized using a nonlinear programming technique in order to obtain an overall optimal performance. Based on the optimal design, a prototype high speed robot is built and tested. The resultant arm design shows an outstanding performance, which is otherwise unattainable if the structure and control are designed separately.