Search

This paper presents a non-parametric image parsing method for segmentation and recognition of roadway assets such as traffic signs, traffic lights, pavement markings, and guardrails from 2D car-mounted video streams. The method can be easily scaled to thousands of video frames captured during data collection and does not need training. Instead, it retrieves a set of most relevant video frames (e.g. highway vs. secondary road) which serve as candidates for superpixel-level annotation. It then obtains superpixels from the video frames and using the retrieval set encodes their visual characteristics using a histogram of different shape, appearance, and color descriptors. Neighborhood contexts are incorporated by using Markov Random Field (MRF) optimization and two types of semantic (e.g. guardrail) and geometric (e.g. horizontal) labels are simultaneously assigned to the superpixels. We introduce a new dataset from I-57 together with its ground truth and present experimental results on both I-57 and SmartRoad datasets. Experimental results with an average accuracy of 88.24% for recognition and 82.02% for segmentation show that our local visual features provide acceptable performance, while the method overall does not require any significant supervised training. This scalable method has potential to reduce the time and effort required for developing road inventories, especially for those such as guardrails and traffic lights that are not typically considered in 2D asset recognition methods. [ABSTRACT FROM AUTHOR]

Efficient data collection of high-quantity and low-cost highway assets such as road signs, traffic signals, light poles, and guardrails is a critical element to the operation, maintenance, and preservation of transportation infrastructure systems. Despite its importance, current practice of highway asset data collection is time-consuming, subjective, and potentially unsafe. The high volume of the data that needs to be collected can also negatively impact the quality of the analysis. To address these limitations, this paper proposes a new algorithm for semantic segmentation and recognition of highway assets using video frames collected from a car-mounted camera. The proposed set of algorithms (1) takes the captured frames and using a pipeline of structure from motion and multiview stereo reconstructs a three-dimensional (3D) point cloud model of the highway and surrounding assets; (2) using a Semantic Texton Forest classifier, each geo-registered two-dimensional (2D) video frame at the pixel-level is segmented based on shape, texture, and color of the highway assets; and finally, (3) based on the results of the 2D segmentation and a new voting scheme, each reconstructed 3D point in the cloud is also categorized for one type of asset and is color coded accordingly. The resulting augmented reality environment that integrates the color-coded point clouds with the geo-registered video frames enables a user to conduct visual walk through and query different categories of assets. Experiments were performed on a challenging video data set containing sequences filmed from a moving car on a 2.2-mi-long, two-lane highway research facility. Experimental results with an average accuracy of 76.50 and 86.75% in segmentation and pixel-level recognition of 12 types of asset categories reflect the promise of the applicability of this approach for segmentation and recognition of highway assets from image-based 3D point clouds. It also enables future algorithmic developments for 3D localization of traffic signs and other assets that are detected using the state-of-the-art vision-based methods. [ABSTRACT FROM AUTHOR]