Back to Search Start Over

Neuromorphic Optical Flow and Real-time Implementation with Event Cameras

Authors :
Schnider, Yannick
Woźniak, Stanislaw
Gehrig, Mathias
Lecomte, Jules
Von Arnim, Axel
Benini, Luca; https://orcid.org/0000-0001-8068-3806
Scaramuzza, Davide; https://orcid.org/0000-0002-3831-6778
Pantazi, Angeliki
Schnider, Yannick
Woźniak, Stanislaw
Gehrig, Mathias
Lecomte, Jules
Von Arnim, Axel
Benini, Luca; https://orcid.org/0000-0001-8068-3806
Scaramuzza, Davide; https://orcid.org/0000-0002-3831-6778
Pantazi, Angeliki
Source :
Schnider, Yannick; Woźniak, Stanislaw; Gehrig, Mathias; Lecomte, Jules; Von Arnim, Axel; Benini, Luca; Scaramuzza, Davide; Pantazi, Angeliki (2023). Neuromorphic Optical Flow and Real-time Implementation with Event Cameras. In: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2023, Vancouver, BC, Canada, 18 June 2023 - 22 June 2023. Institute of Electrical and Electronics Engineers, 4129-4138.
Publication Year :
2023

Abstract

Optical flow provides information on relative motion that is an important component in many computer vision pipelines. Neural networks provide high accuracy optical flow, yet their complexity is often prohibitive for application at the edge or in robots, where efficiency and latency play crucial role. To address this challenge, we build on the latest developments in event-based vision and spiking neural networks. We propose a new network architecture, inspired by Timelens, that improves the state-of-the-art self-supervised optical flow accuracy when operated both in spiking and non-spiking mode. To implement a real-time pipeline with a physical event camera, we propose a methodology for principled model simplification based on activity and latency analysis. We demonstrate high speed optical flow prediction with almost two orders of magnitude reduced complexity while maintaining the accuracy, opening the path for real-time deployments.

Details

Database :
OAIster
Journal :
Schnider, Yannick; Woźniak, Stanislaw; Gehrig, Mathias; Lecomte, Jules; Von Arnim, Axel; Benini, Luca; Scaramuzza, Davide; Pantazi, Angeliki (2023). Neuromorphic Optical Flow and Real-time Implementation with Event Cameras. In: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2023, Vancouver, BC, Canada, 18 June 2023 - 22 June 2023. Institute of Electrical and Electronics Engineers, 4129-4138.
Notes :
application/pdf, info:doi/10.5167/uzh-257367, English, English
Publication Type :
Electronic Resource
Accession number :
edsoai.on1443057788
Document Type :
Electronic Resource