1. Multimodal driver distraction detection using dual-channel network of CNN and Transformer.
- Author
-
Mou, Luntian, Chang, Jiali, Zhou, Chao, Zhao, Yiyuan, Ma, Nan, Yin, Baocai, Jain, Ramesh, and Gao, Wen
- Subjects
- *
TRANSFORMER models , *CONVOLUTIONAL neural networks , *DISTRACTION , *BILEVEL programming , *FEATURE extraction , *TRAFFIC accidents , *DISTRACTED driving - Abstract
Distracted driving has become one of the main contributors to traffic accidents. It is therefore of great interest for intelligent vehicles to establish a distraction detection system that can continuously monitor driver behavior and respond accordingly. Although significant progress has been made in the existing research, most of them focus on extracting either local features or global features while ignoring the other one. To make full use of both local features and global features, we integrate multi-source perception information and propose a novel dual-channel feature extraction model based on CNN and Transformer. In order to improve the model's fitting ability to time series data, the CNN channel and Transformer channel are modeled separately using the mid-point residual structure. The scaling factors in the residual structure are regarded as hyperparameters, and a penalized validation method based on bilevel optimization is introduced to obtain the optimal values automatically. Extensive experiments and comparison with the state-of-the-art methods on a multimodal dataset of driver distraction validate the effectiveness of the proposed method. • A dual-channel network based on CNN and Transformer for feature extraction. • The dual-channel network is modeled using the midpoint residual structure. • A penalized validation method for optimal hyperparameters. • Extensive experiments validate the effectiveness of the proposed method. • It is found that vehicle dynamics data is promising for driver distraction detection. [ABSTRACT FROM AUTHOR]
- Published
- 2023
- Full Text
- View/download PDF