Search

Sensors using ultrasonic sound have proven to provide accurate 3D perception in difficult environments where other modalities fail. Several industrial sectors need accurate and reliable sensing in these harsh conditions. The conventional LiDAR/camera approach in many state-of-the-art autonomous navigation methods is limited to environments with optimal sensing conditions for visual modalities. The use of other sensing modalities can thus improve reliability and usability and increase the application potential of autonomous agents. Ultrasonic measurements provide, compared to LiDAR, a much sparser representation of the environment, making a direct replacement of the LiDAR sensor difficult. In this work, we propose a method to predict LiDAR point cloud data from an in-air acoustic sonar sensor using a convolutional stacked autoencoder. This provides a robotic system with high-resolution measurements and allows for easier integration into existing systems to safely navigate environments where visual modalities become unreliable and less accurate. A video of our predictions is available at https://youtu.be/jlx1S-tslmo.

Array-based imaging sonar sensors often have the advantage that the sampling directions in which sensor data is processed can be arbitrarily chosen. The associated computational cost of the imaging algorithm scales linearly with the number of sampled directions, though, which may limit the use of such sensors for real-time applications such as robotics. This is especially true in case of wide-field sensors, which generally require a larger number of sampled directions (e.g. to cover the full frontal hemisphere). For this reason, it is necessary to optimize the set of sampling directions and reduce its number to a minimum. In this paper we propose a spatial sampling strategy for array based sonar sensors, which takes into account the point-spread function of the array sensor to obtain such a minimal set. We apply this strategy to optimize the set of sampling directions for a specific planar array sonar sensor, and compare the efficiency of the resulting sampling grid to two commonly used sampling strategies.

Pulse-echo sensing is the driving principle behind biological echolocation as well as biologically-inspired sonar and radar sensors. In biological echolocation, a single emitter sends a self-generated pulse into the environment which reflects off objects. A fraction of these reflections are captured by two receivers as echoes, from which information about the objects, such as their position in 3D space, can be deduced by means of timing, intensity and spectral analysis. This is opposed to frequency-modulated continuous-wave radar, which analyses the shift in frequency of the returning signal to determine distance, and requires an array of antenna to obtain directional information. In this work, we present a novel simulator which can generate synthetic pulse-echo measurements for a simulated sensor in a virtual environment. The simulation is implemented by replicating the relevant physical processes underlying the pulse-echo sensing modality, while achieving high performance at update rates above 50 Hz. The system is built to perform design space exploration of sensor hardware and software, with the goals of rapid prototyping and preliminary safety testing in mind. We demonstrate the validity of the simulator by replicating real-world experiments from previous work. In the first case, a subsumption architecture vehicle controller is set to navigate an unknown environment using the virtual sensor. We see the same trajectory pattern emerge in the simulated environment rebuilt from the real experiment, as well as similar activation times for the high-priority behaviors (&plusmn, 1.9%), and low-priority behaviors (&plusmn, 0.2%). In a second experiment, the simulated signals are used as input to a biologically-inspired direct simultaneous mapping and localization (SLAM) algorithm. Using only path integration, 83% of the positional errors are larger than 10 m, while for the SLAM algorithm 95% of the errors are smaller than 3.2m. Additionally, we perform design space exploration using the simulator. By creating a synthetic radiation pattern with increased spatiospectral variance, we are able to reduce the average localization error of the system by 11%. From these results, we conclude that the simulation is sufficiently accurate to be of use in developing vehicle controllers and SLAM algorithms for pulse-echo radar sensors.

State-of-the-art autonomous vehicles mainly rely on optical sensors to perceive their environment. However, the performance of these sensors worsens dramatically in environments where airborne particles are present. Sonar sensors rely on acoustic waves which are able to pass through these distortions. The data gathered from sonar could complement the distorted data of the optical sensors in these earlier mentioned environments. In this paper we will discuss the newest 3D in-air sonar sensor developed by CoSys-Lab: the micro Real Time Imaging Sonar (mu RTIS). It is the smaller version of the previously developed embedded Real Time Imaging Sonar (eRTIS) and consists of a uniform rectangular array of 5 by 6 microphones compared to the pseudo-random 32 microphone array based on Poisson Disc-Sampling used with the eRTIS. In this paper we will discuss the used hardware of the mu RTIS, followed by the implemented processing algorithms to form acoustic images from reflected ultrasonic data. Furthermore, we will discuss the spatial resolution that can be obtained with the presented hardware architecture using different beamforming algorithms. Finally, we will validate the real-life performance, comparing the acoustic images of the mu RTIS when using Delay-And-Sum or MUltiple SIgnal Classification to solve the different directions of arrival.

Filtering relevant signals from noisy sensory input is a crucial challenge for animals [1, 2]. Many bats are acoustic specialists relying on sound to find prey. They discern salient acoustic signals from irrelevant background masking noise. It has long been considered a sensory impossibility for bats to use solely echolocation for the detection of silent and motionless prey resting directly on foliage due to the masking effects of background echoes [3, 4]. Some bats, however, do successfully perform this seemingly impossible task [5], raising the question—what underlying acoustic and behavioral mechanisms do bats use to solve this conundrum? To address this question, we used biomimetic sonar to record high-resolution measurements of echoes from insects resting on leaves. Based on our echo recordings, we predicted optimal approach angles from which masking echoes can best be avoided. In behavioral experiments, we put these predictions to test. We recorded the prey approach behavior of wild bats in a flight cage equipped with an ultrasonic microphone synchronized with two high-speed cameras for 3D flightpath reconstructions. Bats approached prey from our predicted optimal oblique angles, using the leaf as a specular reflector to uncover previously acoustically hidden prey. Our findings disclose key behavioral and acoustic mechanisms enabling the detection of prey echoes that background clutter would otherwise mask. This work adds to the fundamental understanding of how bat echolocation strategies can override acoustic camouflage by silent, motionless prey, thus providing new insights into the evolutionary arms race between predators and their prey.

Obtaining accurate data about the environment in which a robot is located is a crucial matter when it comes to autonomous navigation and other robotic applications. A popular method of acquiring this information is to use sonarrings, where a robot is fitted with multiple simple ultrasound transducers pointed in the directions where an object can appear. However, in a time where accurate 3D data is gaining importance, other sensing modalities are becoming more popular because of the ability to measure dense 3D point clouds. In these point clouds not only the horizontal plane is measured, but objects in the elevation planes can also be registered, which can be very interesting and makes applications such as 3D SLAM or object recognition possible. In this paper we present a way to extract complex 3D point cloud data from the entire surrounding sphere using multiple interconnected eRTIS sensors. These advanced imaging sonar sensors offer the flexibility of the popular sonar-ring in combination with the benefits of some of the competing sensing modalities. The setup presented here uses less sonar sensors (and thus less external hardware) while obtaining more information from the complete frontal hemispheres of each individual sensor. This setup is discussed, along with the issues that arise when using complex imaging sonar sensors in a network, and is tested in an indoor and outdoor environment. At the end of this paper is a discussion of the obtained results.

In this paper we present a flexible low-cost sonar sensor platform that can be used for a wide range of biomimetic sonar experiments and autonomous sonar navigation targeted at robotics applications. The navigation abilities of bats using ultrasound (sonar) in unknown cluttered environments are very effective and can be distilled into a sensor architecture and accompanying control methodology that lends itself to be implemented on cost efficient hardware. The sensor architecture and processing methodology of this sensing platform mimics that of bats. In this paper we specifically focused on the common big-eared bat (Micronycteris microtis) although this could be transferred to other bat species or even other echolocating animals since the experimental platform was designed for flexibility. Using this platform we were able to implement a control system using a subsumption architecture that features different behavior patterns based solely on the sonar sensor as a source of exteroceptive information. In order to validate the combination of our autonomous navigation control system and our developed sonar sensor platform, the hardware was mounted on the P3DX robotics platform that was introduced in an unknown testing environment and have it drive autonomously. These experiments were used to validate our assumption of the efficacy of these relatively simple biomimetic control mechanisms and thus alleviating the need for expensive sensing platforms for certain robotics applications.

Modern three-dimensional (3-D) imaging sonar sensors are perfectly capable of creating awareness of the complete nearby environment with great accuracy. However, as range increases, these systems start to struggle with object detection as the poor attenuation of sound through air causes the signal-to-noise ratio to become too small. When looking at other sensing modalities, such as radar, this problem is tackled by using systems that utilize more than one element in the sending and/or receiving stage resulting in a greater effective detection range. At this further distance, the resolution of the obtained image will tend to decrease. Both radar and underwater sonar systems make use of synthetic aperture systems to increase the spatial imaging resolution. In this letter, we combine both techniques to create an in-air (simultaneous transmit) synthetic aperture beamforming (STSTAB) sensor, based on the eRTIS 3-D sonar sensor, that will increase the system's range and angular resolution without sacrificing the measurement rate. Different aspects of the emitter array will be addressed, such as the optimal placement of the elements of the emitter array, along with the coded signals that will be emitted simultaneously. The result will be a fast-measuring imaging sonar sensor with a narrow point-spread-function and improved range.

Many popular advanced sonar systems provide accurate and reliable measurements containing crucial info needed by robotic applications such as range, bearing and reflection strength of the objects in the field of view. While these sensor systems provide these crucial pieces of information accurately, they are often limited by a lack of processing power and/or size which leads to them needing an external computing device to process all the information generated by the microphone array on the sensor. In this paper we present two versions of a novel fully embedded 3D sonar sensor which have different sensing architectures which enable 3D perception for robotic application in harsh conditions using ultrasound at low cost. Experimental results taken from an office environment will show the 3D localization capabilities and performance of the sensor, showing the sensor has a large field-of-view (FoV) with accurate 3D localization combined with real-time capabilities.

In this paper, we describe the application of a emitter array based on Simultaneous Transmit - Synthetic Aperture Beamforming (ST-STAB) to improve the Signal-to- Noise Ratio (SNR) in an existing, 3D, air-coupled sonar system. The ST-STAB approach allows for the simultaneous emission of orthogonal codes by an array of emitters, which are decoded in the receiver stage, eliminating the need for multiple measurement cycles. By adding more emitters to the sensor system, the overall emitted acoustic energy is increased, while the ST-STAB approach ensures an efficient use of measurement time. We elaborate on the mathematical model of the sensor and signal processing system, provide the associated Point Spread Functions and show experimental validation the constructed prototype array.

Recent advancements in MEMS microphone integration have led to the production of low-cost digital MEMS microphones which are suitable for applications in the ultrasonic acoustic spectrum. Using these microphones instead of their analog counterparts can greatly simplify board- and system-design and reduce overall construction cost of an array built with these sensors. In this paper we will propose an architecture for using these low-cost one-bit MEMS microphones to construct a microphone array and demonstrate their applicability in beamforming-based acoustic imaging algorithms targeted at 3D sonar sensors. We developed a small prototype array do demonstrate that these microphones are well suited for array applications, and the proposed system architecture, developed around an FPGA-SOC, is shown to be ideally suited to interface with a large number of these microphones.

In recent years, our research group has developed state of the art 3D sonar sensors [1]-[3] which use a low-cost MEMS microphone array for real-time acoustic imaging in air. Using this sensor, various robotic applications have been developed [2], including obstacle avoidance and corridor following and SLAM [1]. The developed sensor is capable of localizing an arbitrary number of reflectors, and generates 2D (range versus azimuth) or 3D (range versus azimuth and elevation) acoustic images of the environment by emitting a broadband, spatially omnidirectional acoustic emission. This emission is reflected back by the environment to the microphone array. Using array beamforming algorithms an acoustic image of the environment is created, which subsequently can be either visualized or used in various control algorithms. The overview of the developed sensor array can be seen in figure 1.

Moving a sensor through its environment creates signature time variations of the sensor’s readings often referred to as flow cues. We analyze the acoustic flow field generated by a sonar sensor, capable of imaging the full frontal hemisphere, mounted on a mobile platform. We show how the cues derived from this acoustic flow field can be used directly in a layered control strategy, which supports a robotic platform to perform a set of motion primitives, such as obstacle avoidance, corridor following, and negotiating corners and T-junctions. The programmable nature of the spatial sampling pattern of the sonar allows efficient support of the varying information requirements of the different motion primitives. The proposed control strategy is first validated in a simulated environment and subsequently transferred to a real mobile robot. We present simulated and experimental results on the controller’s performance while executing the different motion primitives. The results further show that the proposed control strategy can easily integrate minimal steering commands given by a user (electric wheelchair application) or by a high-level navigation module (autonomous simultaneous localization and mapping (SLAM) applications).

Modern localization and/or mapping systems based on ultrasound use multiple types of ultrasound transducers. These transducers are used to scan the environment and create situational awareness so that obstacles are spotted and collisions may be prevented. Most commercially available transducers however have a field of view (FOV) that is too limited for these applications. As a result of this, multiple transducers are necessary to get a decent perceptional coverage of the environment. The signals coming from the different transducers can cause unnecessary complications. In this paper, we propose a method for widening the FOV of off-the-shelf ultrasonic transducers by creating a baffle that fits on these transducers. This will significantly improve the process of mapping the environment for a wide range of applications, such as obstacle avoidance and SLAM. We provide strong experimental evidence for the widening of the directivity patterns by means of various 3-D-printed baffles.

A novel biomimetic sonarhead has been developed to allow researchers of bat echolocation behavior and biomimetic sonar to perform experiments with a system similar to the bat's sensory system. The bat's echolocation-related transfer function (ERTF) is implemented using an array of receivers to implement the head-related transfer function (HRTF), and an array of emitters mounted on a cylindrical manifold to implement the emission pattern of the bat. The complete system is controlled by a field-programmable gate array (FPGA) based embedded system connected through a USB interface.

Echolocating bats have excellent spatial memory and are able to navigate to salient locations using bio-sonar. Navigating and route-following require animals to recognize places. Currently, it is mostly unknown how bats recognize places using echolocation. In this paper, we propose template based place recognition might underlie sonar-based navigation in bats. Under this hypothesis, bats recognize places by remembering their echo signature - rather than their 3D layout. Using a large body of ensonification data collected in three different habitats, we test the viability of this hypothesis assessing two critical properties of the proposed echo signatures: (1) they can be uniquely classified and (2) they vary continuously across space. Based on the results presented, we conclude that the proposed echo signatures satisfy both criteria. We discuss how these two properties of the echo signatures can support navigation and building a cognitive map. DOI: http://dx.doi.org/10.7554/eLife.14188.001, eLife digest Bats produce loud calls and listen to the returning echoes to find their way around. This process, known as echolocation, is sometimes described as 'seeing with sound'. The way bats perceive the world through echolocation, however, is fundamentally different from how we experience it through vision. Echolocation provides much less information about the world than vision does, but despite this, bats are agile navigators and hunters. It is not clear how bats navigate so well without much information. In particular, researchers would like to know how echolocating bats recognize the places that they regularly visit while foraging and navigating. When we visually recognize places, we identify and localize the various objects making up the scene. But echolocation is unlikely to provide enough information to allow bats to identify and localize the objects in a particular place. To investigate how bats recognize places, Vanderelst et al. built an artificial bat: a device that contained ultrasonic microphones to act like the bat’s ears and an ultrasonic speaker to act like the bat’s mouth. The artificial bat device was then used to collect echoes from different locations in real bat habitats. Processing the echoes using machine-learning techniques showed that the echoes that returned from each location were different enough for a computer to recognize the location. By using a simplified version of the echoes, Vanderelst et al. also showed that the locations could be recognized even if there was not enough information to identify specific objects or vegetation at the site. This suggests that bats do not simply use echolocation to recreate the three-dimensional layout of a location, as some researchers have proposed. While much remains to be learned about how bats use echolocation for navigation, future work that teases out bat navigation strategies might help us to build robots that can navigate using similar tactics. DOI: http://dx.doi.org/10.7554/eLife.14188.002

In this paper, we present a solution to the Simultaneous Localization and Mapping problem by combining a novel 3D in-air sonar sensor with techniques for estimating the ego-motion of a mobile platform (called acoustic flow odometry) and a bio-inspired mapping module (called BatSLAM). This combination eliminates the need for odometric information originating from the robot's motor controller, enabling applications where that information is difficult to obtain, e. g. many electric wheelchairs. The proposed method exploits the programmable spatial sampling capabilities of the 3D sonar system to derive from the same set of received echo signals both an accurate representation of reflectors in the horizontal plane (2D energyscape) and a more coarse representation of reflectors in the frontal hemisphere (3D energyyscape). Using a mapping experiment we demonstrate that the motion estimates originating from the acoustic flow module combined with the coarse frontal hemisphere data provide sufficient information for the mapping module to reconstruct the robot's trajectory.

Recently we have developed an in-air sonar system that is capable of generating 3D descriptions of environments containing multiple reflectors using a single broadband emission in combination with a microphone array. We have coined the term "energyscape" for the representation of the environment, i.e. reflected energy as a function of range and direction, generated by this system. While such rich sensor data can prove useful without object segmentation for certain applications, it is sometimes desirable to map the energyscapes onto individual reflector positions. Simple ad-hoc segmentation methods often have problems with the high dynamic range in the sensor data. We propose a method using sparse decomposition of the energyscapes using a dictionary containing direction dependent Point Spread Functions (PSF) of the sensor system. Using an L1-regularized least squares solution method we demonstrate reconstruction of actual measured energyscapes and estimation of reflector positions in complex scenarios.

We propose to combine a biomimetic navigation model which solves a simultaneous localization and mapping task with a biomimetic sonar mounted on a mobile robot to address two related questions. First, can robotic sonar sensing lead to intelligent interactions with complex environments? Second, can we model sonar based spatial orientation and the construction of spatial maps by bats? To address these questions we adapt the mapping module of RatSLAM, a previously published navigation system based on computational models of the rodent hippocampus. We analyze the performance of the proposed robotic implementation operating in the real world. We conclude that the biomimetic navigation model operating on the information from the biomimetic sonar allows an autonomous agent to map unmodified (office) environments efficiently and consistently. Furthermore, these results also show that successful navigation does not require the readings of the biomimetic sonar to be interpreted in terms of individual objects/landmarks in the environment. We argue that the system has applications in robotics as well as in the field of biology as a simple, first order, model for sonar based spatial orientation and map building.

Array beamforming techniques allow for the generation of 3-D spatial filters which can be used to localize objects in a large field of view (FOV) without the need for mechanical scanning. By combining broadband beamforming with a sparse, random array of receivers, we have constructed a low-cost, yet powerful, in-air sonar system, which is suited for a wide range of robotic applications. Experimental results in unmodified office environments show the performance of the sonar sensor. In particular, we document the sensor's capacity to produce 3-D location measurements in the presence of multiple highly overlapping echoes. We show how this capability makes possible the combination of a wide FOV with accurate 3-D localization, allowing the sensor to operate under real-time constraints in realistic environments. To demonstrate the use of this sensor, we describe an odometry application that estimates egomotion of a mobile robot using acoustic flow.

We present a biomimetic sonar system consisting of a single emitter and two receivers endowed with realistic spatial filter characteristics by inserting them in plastic replicas of bat pinnae. The outputs of these receivers are processed by a simple functional model of the processing performed in the cochlea. We show that these cochleograms contain sufficient amounts of information for the robust localization of a robot in unmodified office environments. This localization component is part of a more general biomimetic Simultaneous Localization and Mapping system (SLAM) for an autonomous robot based on a model of the rodent Hippocampus and the use of sonar instead of vision as exteroceptive sensory input (BatSLAM). To quantify the performance of the biomimetic sonar system in the robot localization task, and to be able to quantitatively compare the performance of different types of (sonar) sensors, we propose a technique based on conditional entropy.

Biomimetic sonar systems, i.e. sonar systems making use of spectral cues for the localization of one or more reflectors, depend heavily on the spatial filters of the reception subsystem. These spatial filters can be implemented in two ways, e. g., by means of an artificial pinna or by means of an array of microphones in combination with a beamforming algorithm. In this paper we compare two such systems using an information theoretic model, which allows objective evaluation of each system from an echolocation point of view.

In this study we investigate the effects of medium disturbances on the usability of a biomimetic sonar system in a target localization application. In particular, we concentrate on a fire-fighting application giving rise to specific types of medium disturbances. We determine experimentally the effects of these medium disturbances: undisturbed air (reference), the presence of small water droplets, large convective air-currents and flames, on the received echo spectra. As the echo spectra are used to estimate the target location, these different conditions by disturbing to varying extents sound propagation through the medium will have an effect on the sonar system's localization performance. Using an information theoretic model of the spectrum based localization mechanism we characterize the loss in localization accuracy due to the different medium disturbances. We conclude that an echolocation system based on the sonar system of the bat Phyllostomus discolor allows for robust localization even in highly disturbed media. However, while the need for replacing the emission pattern of the bat by that of a standard transducer in the biomimetic sonar system makes little difference in the ‘reference’ and ‘water spray’ conditions it limits echolocation performance in the ‘air-currents’ condition and makes the use of the proposed system very difficult in the ‘flames’ condition.

In recent years, our research group has developed state of the art 3D sonar sensors [1] , [2] which use a low-cost MEMS microphone array for real-time acoustic imaging in air. Using this sensor, various robotic applications have been de-veloped [1] , including obstacle avoidance, corridor following and SLAM. The developed sensor is capable of localizing an arbitrary number of reflectors, and generates 2D (range versus azimuth) or 3D (range versus azimuth and elevation) acoustic images of the environment by emitting a broadband, spatially omnidirectional acoustic emission. This emission is reflected back by the environment to the microphone array. Using array beamforming algorithms an acoustic image of the environment is created, which subsequently can be either visualized or used in various control algorithms. The overview of the developed sensor array can be seen in figure 1 .