Drone Detection Using Deep Learning and Sensor Fusion

We present a combined approach using computer vision and radio frequencies to detect and confirm the presence of an Unmanned Aerial Vehicle to avoid false positives classifications in the real-time classification of Unmanned Aerial Vehicles. Current state of the art implementations for drone detection are reliant on a single sensor to detect drones and often produces false positives as the sensor fails. These false positive results arise from similarities between either bio-life creatures and drones or sub-optimal visual conditions challenging the classification of these drones due to the difficulty in distinguishing between background. When detecting through optimal or sub-optimal visual conditions (like occlusion), the radio frequencies emitted from the various states of the drone will stay consistent allowing for the combination of both visual and radio frequencies to be utilized for the classification of drones. Using this combined approach of computer vision and radio frequencies will also improve detection in cases of bad weather or visibility issues where the drone will be difficult to distinguish in fog, rain, and other weather conditions. We propose an artificial neural network-based detection system which uses a deep neural network to process the RF data and convolutional neural network to process the image data. The features from these networks are inputted into another deep neural network which outputs a single probability prediction of drone presence. This allows for a sensor fusion approach where the artificial neural network relies on both the image and the RF input in order to make a prediction. Due to the increase in technological advances with commercial Unmanned Aerial Vehicle (UAV) the need to monitor and classify these drones is increasing. Drone technology is gradually becoming cheaper and easier to obtain, the danger of misuse and hostile use of drones are also on the rise. Sites like airports are at risk if these drones go undetected. Current systems usually use a single sensor (camera/radar) to detect these drones which are not always effective. We implemented a system which effectively uses RF and camera data to improve detection results.