Back to Search
Start Over
Fusing RGB and Thermal Imagery with Channel State Information for Abnormal Activity Detection Using Multimodal Bidirectional LSTM
- Source :
- Cyber-Physical Security for Critical Infrastructures Protection
- Publication Year :
- 2021
-
Abstract
- In this paper, we present a multimodal deep model for detection of abnormal activity, based on bidirectional Long Short-Term Memory neural networks (LSTM). The proposed model exploits three different input modalities: RGB imagery, thermographic imagery and Channel State Information from Wi-Fi signal reflectance to estimate human intrusion and suspicious activity. The fused multimodal information is used as input in a Bidirectional LSTM, which has the benefit of being able to capture temporal interdependencies in both past and future time instances, a significant aspect in the discussed unusual activity detection scenario. We also present a Bayesian optimization framework that fine-tunes the Bidirectional LSTM parameters in an optimal manner. The proposed framework is evaluated on real-world data from a critical water infrastructure protection and monitoring scenario and the results indicate a superior performance compared to other unimodal and multimodal approaches and classification models.
Details
- Language :
- English
- Volume :
- 12618
- Database :
- OpenAIRE
- Journal :
- Cyber-Physical Security for Critical Infrastructures Protection
- Accession number :
- edsair.pmc...........75f6c75a2378a4d9b91e2f5557a3134a