Search

Computational neuromorphic imaging (CNI) with event cameras offers advantages such as minimal motion blur and enhanced dynamic range, compared to conventional frame-based methods. Existing event-based radiance field rendering methods are built on neural radiance field, which is computationally heavy and slow in reconstruction speed. Motivated by the two aspects, we introduce Ev-GS, the first CNI-informed scheme to infer 3D Gaussian splatting from a monocular event camera, enabling efficient novel view synthesis. Leveraging 3D Gaussians with pure event-based supervision, Ev-GS overcomes challenges such as the detection of fast-moving objects and insufficient lighting. Experimental results show that Ev-GS outperforms the method that takes frame-based signals as input by rendering realistic views with reduced blurring and improved visual quality. Moreover, it demonstrates competitive reconstruction quality and reduced computing occupancy compared to existing methods, which paves the way to a highly efficient CNI approach for signal processing.

Neuromorphic imaging is a bio-inspired technique that imitates the human retina to sense variations in a dynamic scene. It responds to pixel-level brightness changes by asynchronous streaming events and boasts microsecond temporal precision over a high dynamic range, yielding blur-free recordings under extreme illumination. Nevertheless, such a modality falls short in spatial resolution and leads to a low level of visual richness and clarity. Pursuing hardware upgrades is expensive and might cause compromised performance due to more burdens on computational requirements. Another option is to harness offline, plug-in-play neuromorphic super-resolution solutions. However, existing ones, which demand substantial sample volumes for lengthy training on massive computing resources, are largely restricted by real data availability owing to the current imperfect high-resolution devices, as well as the randomness and variability of motion. To tackle these challenges, we introduce the first self-supervised neuromorphic super-resolution prototype. It can be self-adaptive to per input source from any low-resolution camera to estimate an optimal, high-resolution counterpart of any scale, without the need of side knowledge and prior training. Evaluated on downstream event-driven tasks, such a simple yet effective method can obtain competitive results against the state-of-the-arts, significantly promoting flexibility but not sacrificing accuracy. It also delivers enhancements for inferior natural images and optical micrographs acquired under non-ideal imaging conditions, breaking through the limitations that are challenging to overcome with traditional frame techniques. In the current landscape where the use of high-resolution cameras for event-based sensing remains an open debate, our solution serves as a cost-efficient and practical alternative, paving the way for more intelligent imaging systems., Comment: 11 pages, 13 figures, and 3 tables

The Shack-Hartmann wavefront sensor is widely employed in adaptive optics systems to measure optical aberrations. However, simultaneously achieving high sensitivity and large dynamic range is still challenging, limiting the performance of diagnosing fast-changing turbulence. To overcome this limitation, we propose neuromorphic Shack-Hartmann wave normal sensing (NeuroSH). NeuroSH is a unifying framework that harnesses the computational neuromorphic imaging paradigm to extract the high-dimensional wave normal from temporal diversity measurements. Both numerical analysis and experimental verification demonstrate the feasibility of NeuroSH. To the best of our knowledge, the proposed NeuroSH is the first scheme to surpass the optical dynamic range limitation under challenging dynamic scenarios, thereby advancing ultra-fast turbulence mitigation technology for cutting-edge imagers.

Bio-inspired neuromorphic cameras sense illumination changes on a per-pixel basis and generate spatiotemporal streaming events within microseconds in response, offering visual information with high temporal resolution over a high dynamic range. Such devices often serve in surveillance systems due to their applicability and robustness in environments with high dynamics and harsh lighting, where they can still supply clearer recordings than traditional imaging. In other words, when it comes to privacy-relevant cases, neuromorphic cameras also expose more sensitive data and pose serious security threats. Therefore, asynchronous event streams necessitate careful encryption before transmission and usage. This work discusses several potential attack scenarios and approaches event encryption from the perspective of neuromorphic noise removal, in which we inversely introduce well-crafted noise into raw events until they are obfuscated. Our evaluations show that the encrypted events can effectively protect information from attacks of low-level visual reconstruction and high-level neuromorphic reasoning, and thus feature dependable privacy-preserving competence. The proposed solution gives impetus to the security of event data and paves the way to a highly encrypted technique for privacy-protective neuromorphic imaging., Comment: 6 pages, 6 figures, and 2 tables. Accepted by IOP Neuromorphic Computing and Engineering

Scattering medium brings great difficulties to locate and image planar objects especially when the object has a large depth. In this letter, a novel learning-based method is presented to locate and image the object hidden behind a thin scattering diffuser. A multi-task network, named DINet, is constructed to predict the depth and the image of the hidden object from the captured speckle patterns. The provided experiments verify that the proposed method enables to locate the object with a depth mean error less than 0.05 mm, and image the object with an average PSNR above 24 dB, in a large depth ranging from 350 mm to 1150 mm. The constructed DINet can obtain multiple physical information via a single speckle pattern, including both the depth and image. Comparing with the traditional methods, it paves the way to the practical applications requiring large imaging depth of field behind scattering media.

Strong scattering medium brings great difficulties to optical imaging, which is also a problem in medical imaging and many other fields. Optical memory effect makes it possible to image through strong random scattering medium. However, this method also has the limitation of limited angle field-of-view (FOV), which prevents it from being applied in practice. In this paper, a kind of practical convolutional neural network called PDSNet is proposed, which effectively breaks through the limitation of optical memory effect on FOV. Experiments is conducted to prove that the scattered pattern can be reconstructed accurately in real-time by PDSNet, and it is widely applicable to retrieve complex objects of random scales and different scattering media., Comment: 15 pages with 9 figures