Search

With the ever-increasing integration and parallel computing capabilities of 3D processor arrays, the occurrence of processor elements (PEs) failures caused by various factors has become more prevalent. Therefore, the implementation of a fault-tolerant mechanism that uses the remaining fault-free PEs to reconfigure sub-array becomes critical. In this paper, we study the problem of reconfiguring a 3D subarray with as many fault-free PEs as possible, which has been shown to be NP-complete in previous work. Although prior algorithms have been effective under low fault densities, they are severely limited when faced with high fault densities. To address this, we first define the bottleneck of the 3D processor array, proposed a novel method to identify the physical bottleneck plane that restricts the reconfigurable size of the logical sub-array and prove its correctness. Then, we propose an effective compensation strategy that can fully utilize the fault-free PEs in the bottleneck plane. Under this strategy, a sliding-window weight calculation method is proposed to determine the priority of compensation. Finally, we proposed a heuristic algorithm, which can construct the maximum target array from different dimensions in polynomial time. Experimental results demonstrate that the proposed algorithm exhibits favorable performance in terms of harvest and degradation. For the random-failure model, the improvement in the harvest for fault-free PEs is up to 32.03% on a $32 \times 32 \times 32$32×32×32 host array with a 20% fault density. And for the clustered fault model, the improvement in harvest is up to 70.63% on a $32 \times 32 \times 32$32×32×32 host array distributed with 12 cluster failures of size $6 \times 6 \times 6$6×6×6.

With the progressive augmentation of the density of 3-D processor arrays, some processor elements (PEs) often fail due to overload or overheating during massively parallel computing operations. Therefore, it is necessary to take effective fault-tolerant technology to ensure the reliability of the system. This article investigates an efficient reconfiguration method to construct 3-D fault-free logical subarray with more fault-free PEs and less interconnection length (interlength). First, we propose a novel method based on the barrel effect to find the bottleneck plane of 3-D processor arrays. Second, an efficient compensation strategy is proposed to replace faulty PEs on adjacent physical planes with fault-free PEs on the bottleneck planes, which leads to more fault-free PEs that can be used to construct the subarray. Then, we propose a heuristic to construct the subarray and optimize iteration redundancy to accelerate reconstruction. Finally, a heuristic optimization algorithm is proposed to reduce the interlength between PEs, which can reduce the dynamic power consumption and communication costs. In addition, we propose a more accurate method to calculate the lower bound of the interlength to better evaluate the performance of the algorithm. Simulation experiments show that, compared to the state-of-the-arts, on $128\times 128\times 128$ host array, the utilization rate of fault-free PEs can be improved up to 15.6% and the interlength redundancy can be reduced by 78.2% for random faults. On $64\times 64\times 64$ host array, the average improvement of the two indicators under clustered faults can reach 93.2% and 69.3%. Moreover, for all cases considered, the proposed new lower bound and reconstruction time can be reduced by an average of 18.47% and 76.13%, respectively.

Deep learning has achieved great success in computer vision, especially in image classification tasks. How to improve the generalization ability and compactness of deep neural networks has gradually attracted widespread attention from researchers. Knowledge distillation is an effective technique for model compression. It transfers general knowledge from a sophisticated teacher model to a smaller student model. Recently, some studies refine knowledge from feature maps or adopt complex attention mechanisms to better supervise students imitating teachers. However, their methods focus too much on how to improve students’ accuracy and largely overlook the associated training costs, which violates the original intention of knowledge distillation to compress the model. To achieve a balance between performance and efficiency, in this paper, we introduce a straightforward and effective distillation method to utilize the deepest feature maps to enhance shallow features. Specifically, our method performs processing only on the original feature maps without an extra assisting network. Moreover, we use cross-layer feature fusion to enhance the attention on shallow feature maps. By visualizing the features of different layers, we demonstrate the importance of the fusion operation in our method. Our experimental results on the CIFAR-100, tinyImageNet and miniImageNet datasets show that our approach outperforms previous methods, especially in the balance between performance and training cost. Further ablative studies verify the effectiveness of the design.

Since deep learning methods have achieved great success in both computer vision and natural language processing, video captioning tasks based on these two fields have also attracted extensive attention. Video captioning is a challenging task, which aims to present video information in the form of natural language to enhance video intelligibility. Most of the current researches in video captioning focus on the behavioral description of the main objects of the video, especially on the holistic understanding of the content. This trend makes most video captioning efforts ignoring the characteristics of smaller objects in the video, resulting in ambiguous, imprecise, or even fundamentally wrong descriptions. In this paper, a novel video captioning method MSLR is proposed, which improves the accuracy of video description by extracting features of video objects with different granularity and preserving long-range temporal dependencies. Specifically, the proposed method performs convolution operations at different scales to obtain different granular spatial features of videos and then fuses them to generate a unified spatial representation. On this basis, a temporal extraction network is further constructed using non-local blocks to preserve the long-range dependencies of videos. Evaluated on two popular benchmark datasets, the experimental results demonstrate the superiority of MSLR over the previous state-of-the-art methods, and the effectiveness of MSLR components is verified through ablation experiments and text evaluation.

Self-attention-based image captioning model exists visual features’ spatial information loss problem, introducing relative position encoding can solve the problem to some extent. However, it will bring additional parameters and greater computational complexity. To solve the above problem, we propose a novel local–global MLFormer (LG-MLFormer) with specifically designed encoder module Local–global multi-layer perceptron (LG-MLP). The LG-MLP can capture the latent correlations between different images and its linear stacking calculation mode can reduce computational complexity. It consists of two independent local MLP (LM) modules and a cross-domain global MLP (CDGM) module. The LM specially designs the mapping dimension between linear layers to realize the self-compensation of visual features’ spatial information without introducing relative position encoding. The CDGM module aggregates cross-domain potential correlations between grid-based features and region-based features to realize the complementary advantages of these global and local semantic associations. Experiments on the Karpathy test split and the online test server reveal that our approach provides superior or comparable performance to the state-of-the-art (SOTA). Trained models and code for reproducing the experiments are publicly available at: https://github.com/wxx1921/LGMLFormer-local-and-global-mlp-for-image-captioning.