Adding ears to intelligent connected vehicles by combining microphone arrays and high definition map

Abstract Ears are responsible for the sense of hearing, which is indispensable for a human to understand the environment. It is similar to an intelligent vehicle where acoustical data contains useful and valuable information to comprehend the environment to enable autonomous driving. One important application of acoustical data processing is to localize sound sources. However, due to the complexity of microphone array setup and the unstable localization performance against noises, few research articles tried to localize surrounding vehicles based on on‐board microphone arrays. The main contribution is the first implementation to combine roof‐mounted microphones and high definition maps as vehicle' s ears for online 3D localization of surround vehicles. This article designed a six‐microphones sensor array dedicated to the on‐board application and improved sound source localization algorithms using the observer technique to adapt to dynamic driving scenarios. A map‐based localization enhancement method is also proposed to make the system more robust to noises and model errors. Feasibility tests are conducted with real vehicles in the scenarios of pass‐by and overtaking. Experimental results validate the feasibility of using microphones to localize surrounding vehicles with lane‐level accuracy.