FPGA-Based CNN for Real-Time UAV Tracking and Detection

Neural Networks (NNs) are now being extensively utilised in various Artificial Intelligence platforms specifically in the area of image classification and real-time object tracking. We propose a novel design to address the problem of real time Unmanned Aerial Vehicle (UAV) monitoring and detection using a Zynq Ultrascale FPGA-based Convolutional Neural Network (CNN) implementation. The biggest challenge while implementing real-time algorithms on FPGAs is the limited DSP hardware resources available on FPGA platforms. Our proposed design overcomes the challenge of autonomous real time UAV detection and tracking on a Xilinx's Zynq Ultrascale XCZU9EG System on a Chip (SoC) platform. Our proposed design explores and provides a solution for overcoming the challenge of limited floating-point resources, whilst maintaining real-time performance. The solution consists of two modules: the UAV tracking and the neural network based UAV detection module. The tracking module uses our novel background-differencing algorithm whilst the UAV detection is based on a modified CNN algorithm, designed to give the maximum Field-Programmable Gate Array (FPGA) performance. These two modules are designed to complement each other and are enabled simultaneously to provide an enhanced real-time UAV detection for any given video input. The proposed system has been tested on detecting real-life flying UAVs, achieving an accuracy of 82\%, running at the full frame rate of the input camera for both tracking and Neural Network (NN) detection, achieving similar performance than an equivalent Deep Learning Processor Unit (DPU) Ultrascale FPGA based HD video and tracking implementation, but with lower resource utilisation as shown by our results.