Search

Generally, the recognition performance of lightweight models is often lower than that of large models. Knowledge distillation, by teaching a student model using a teacher model, can further enhance the recognition accuracy of lightweight models. In this paper, we approach knowledge distillation from the perspective of intermediate feature-level knowledge distillation. We combine a cross-stage feature fusion symmetric framework, an attention mechanism to enhance the fused features, and a contrastive loss function for teacher and student models at the same stage to comprehensively implement a multistage feature fusion knowledge distillation method. This approach addresses the problem of significant differences in the intermediate feature distributions between teacher and student models, making it difficult to effectively learn implicit knowledge and thus improving the recognition accuracy of the student model. Compared to existing knowledge distillation methods, our method performs at a superior level. On the CIFAR100 dataset, it boosts the recognition accuracy of ResNet20 from 69.06% to 71.34%, and on the TinyImagenet dataset, it increases the recognition accuracy of ResNet18 from 66.54% to 68.03%, demonstrating the effectiveness and generalizability of our approach. Furthermore, there is room for further optimization of the overall distillation structure and feature extraction methods in this approach, which requires further research and exploration. [ABSTRACT FROM AUTHOR]

Wearable sensors are widely used in medical applications and human–computer interaction because of their portability and powerful privacy. Human activity identification based on sensor data plays a vital role in these fields. Therefore, it is important to improve the recognition performance of different types of actions. Aiming at the problems of insufficient time-varying feature extraction and gradient explosion caused by too many network layers, a time convolution network recognition model with attention mechanism (TCN-Attention-HAR) was proposed. The model effectively recognizes and emphasizes the key feature information. The ability of extracting temporal features from TCN (temporal convolution network) is improved by using the appropriate size of the receiver domain. In addition, attention mechanisms are used to assign higher weights to important information, enabling models to learn and identify human activities more effectively. The performance of the Open Data Set (WISDM, PAMAP2 and USC-HAD) is improved by 1.13%, 1.83% and 0.51%, respectively, compared with other advanced models, these results clearly show that the network model presented in this paper has excellent recognition performance. In the knowledge distillation experiment, the parameters of student model are only about 0.1% of those of teacher model, and the accuracy of the model has been greatly improved, and in the WISDM data set, compared with the teacher's model, the accuracy is 0.14% higher. [ABSTRACT FROM AUTHOR]

The presence of haze results in diminished image clarity, the loss of image details, color alterations, and a decline in image visibility. This paper proposes an efficient multi-stage feature fusion defogging network based on the attention mechanism for effective defogging in complex environments. To create a multi-branch defogging network, the model integrates several attention techniques to create various sorts of branching network architectures. The image's spatial details and contextual information are supplemented and merged based on the recovered image feature information of different network branches to increase the effectiveness of the network model. The stage attention fusion mechanism created among several network branches can lessen the loss of image data during feature extraction and improve the effectiveness of the image-defogging operation. The experimental results demonstrate that the proposed algorithm has superior defogging performance in both synthetic and real-world scene datasets and performs more admirably in terms of accuracy compared to other sophisticated algorithms, particularly in the RESIDE and O-Haze datasets. The PSNR metrics on the RESIDE and O-Haze datasets are improved by 1.58 dB and 1.61 dB, respectively, compared to the best-advanced technique suggested in this study, and the SSIM metrics on the O-Haze dataset are improved by 3.4%. [ABSTRACT FROM AUTHOR]