Search

Segmentation methods based on convolutional neural networks (CNN) have achieved remarkable results in the field of medical image segmentation due to their powerful representation capabilities. However, for brain-tumor segmentation, owing to the significant variations in shape, texture, and location, traditional convolutional neural networks (CNNs) with limited convolutional kernel-receptive fields struggle to model explicit long-range (global) dependencies, thereby restricting segmentation accuracy and making it difficult to accurately identify tumor boundaries in medical imaging. As a result, researchers have introduced the Swin Transformer, which has the capability to model long-distance dependencies, into the field of brain-tumor segmentation, offering unique advantages in the global modeling and semantic interaction of remote information. However, due to the high computational complexity of the Swin Transformer and its reliance on large-scale pretraining, it faces constraints when processing large-scale medical images. Therefore, this study addresses this issue by proposing a smaller network, consisting of a dual-encoder network, which also resolves the instability issue that arises in the training process of large-scale visual models with the Swin Transformer, where activation values of residual units accumulate layer by layer, leading to a significant increase in differences in activation amplitudes across layers and causing model instability. The results of the experimental validation using real data show that our dual-encoder network has achieved significant performance improvements, and it also demonstrates a strong appeal in reducing computational complexity.

The main objective of scene text recognition is to recognize text in complex images and convert it into editable text. However, scene text recognition research has long been focused on English, and there is a lack of research on other small languages, such as Uyghur language. This paper conducts a series of studies on Uyghur text recognition in natural scenes, with the following main contributions: 1) To address the lack of Uyghur scene text recognition datasets, we established the Synthetic Uyghur Scene Text dataset SUST and the Real Uyghur Scene Text dataset RUST, and we augmented RUST with STR-Aug; 2) The contemporary existing STR models are selected to conduct experiments on the dataset proposed in this paper, through which we search for the model structure suitable for Uyghur scene text recognition; 3) According to the characteristics of Uyghur text, this paper designs a lightweight Uyghur text recognition model named LUSTR, which takes into account both lightweight and accuracy, and achieves good performance in the Uyghur scene text recognition dataset proposed in this paper. The SUST and RUST are now publicly available at https://github.com/kongfnajie/SUST-and-RUST-datasets-for-Uyghur-STR.

Aiming at the task of automatic brain tumor segmentation, this paper proposes a new DenseTrans network. In order to alleviate the problem that convolutional neural networks(CNN) cannot establish long-distance dependence and obtain global context information, swin transformer is introduced into UNet++ network, and local feature information is extracted by convolutional layer in UNet++. then, in the high resolution layer, shift window operation of swin transformer is utilized and self-attention learning windows are stacked to obtain global feature information and the capability of long-distance dependency modeling. meanwhile, in order to alleviate the secondary increase of computational complexity caused by full self-attention learning in transformer, deep separable convolution and control of swin transformer layers are adopted to achieve a balance between the increase of accuracy of brain tumor segmentation and the increase of computational complexity. on BraTs2021 data validation set, model performance is as follows: the dice dimilarity score was 93.2%,86.2%,88.3% in the whole tumor,tumor core and enhancing tumor, hausdorff distance(95%) values of 4.58mm,14.8mm and 12.2mm, and a lightweight model with 21.3M parameters and 212G flops was obtained by depth-separable convolution and other operations. in conclusion, the proposed model effectively improves the segmentation accuracy of brain tumors and has high clinical value.

IntroductionRecently, the Transformer model and its variants have been a great success in terms of computer vision, and have surpassed the performance of convolutional neural networks (CNN). The key to the success of Transformer vision is the acquisition of short-term and long-term visual dependencies through self-attention mechanisms; this technology can efficiently learn global and remote semantic information interactions. However, there are certain challenges associated with the use of Transformers. The computational cost of the global self-attention mechanism increases quadratically, thus hindering the application of Transformers for high-resolution images.MethodsIn view of this, this paper proposes a multi-view brain tumor segmentation model based on cross windows and focal self-attention which represents a novel mechanism to enlarge the receptive field by parallel cross windows and improve global dependence by using local fine-grained and global coarse-grained interactions. First, the receiving field is increased by parallelizing the self-attention of horizontal and vertical fringes in the cross window, thus achieving strong modeling capability while limiting the computational cost. Second, the focus on self-attention with regards to local fine-grained and global coarse-grained interactions enables the model to capture short-term and long-term visual dependencies in an efficient manner.ResultsFinally, the performance of the model on Brats2021 verification set is as follows: dice Similarity Score of 87.28, 87.35 and 93.28%; Hausdorff Distance (95%) of 4.58 mm, 5.26 mm, 3.78 mm for the enhancing tumor, tumor core and whole tumor, respectively.DiscussionIn summary, the model proposed in this paper has achieved excellent performance while limiting the computational cost.

Low-resource languages often face the problem of insufficient data, which leads to poor quality in machine translation. One approach to address this issue is data augmentation. Data augmentation involves creating new data by transforming existing data through methods such as flipping, cropping, rotating, and adding noise. Traditionally, pseudo-parallel corpora are generated by randomly replacing words in low-resource language machine translation. However, this method can introduce ambiguity, as the same word may have different meanings in different contexts. This study proposes a new approach for low-resource language machine translation, which involves generating pseudo-parallel corpora by replacing phrases. The performance of this approach is compared with other data augmentation methods, and it is observed that combining it with other data augmentation methods further improves performance. To enhance the robustness of the model, R-Drop regularization is also used. R-Drop is an effective method for improving the quality of machine translation. The proposed method was tested on Chinese–Kazakh (Arabic script) translation tasks, resulting in performance improvements of 4.99 and 7.7 for Chinese-to-Kazakh and Kazakh-to-Chinese translations, respectively. By combining the generation of pseudo-parallel corpora through phrase replacement with the application of R-Drop regularization, there is a significant advancement in machine translation performance for low-resource languages.

Scene text recognition (STR) has been a hot research field in computer vision, aiming to recognize text in natural scenes using computers. Currently, attention-based encoder–decoder frameworks struggle to precisely align feature regions with the target object when dealing with complex and low-quality images, a phenomenon known as attention drift. Additionally, with the rise of Transformer, the increasing size of parameters results in higher computational costs. In order to solve the above problems, based on the latest research results of Vision Transformer (ViT), we utilize an additional position-enhancement branch to alleviate attention drift and dynamically fused position information with visual information to achieve better recognition accuracy. The experimental results demonstrate that our model achieves a 3% higher average recognition accuracy on the test set compared to the baseline. Meanwhile, our model maintains the advantage of a small number of parameters and fast inference speed, achieving a good balance between accuracy, speed, and computational load.

VQA (Visual Question Answering) is a multi-model task. Given a picture and a question related to the image, it will determine the correct answer. The attention mechanism has become a de facto component of almost all VQA models. Most recent VQA approaches use dot-product to calculate the intra-modality and inter-modality attention between visual and language features. In this paper, the BAN (Bilinear Attention Network) method was used to calculate attention. We propose a deep multimodality bilinear attention network (DMBA-NET) framework with two basic attention units (BAN-GA and BAN-SA) to construct inter-modality and intra-modality relations. The two basic attention units are the core of the whole network framework and can be cascaded in depth. In addition, we encode the question based on the dynamic word vector of BERT(Bidirectional Encoder Representations from Transformers), then use self-attention to process the question features further. Then we sum them with the features obtained by BAN-GA and BAN-SA before the final classification. Without using the Visual Genome datasets for augmentation, the accuracy of our model reaches 70.85% on the test-std dataset of VQA 2.0.

Visual question answering (VQA) requires a deep understanding of images and their corresponding textual questions to answer questions about images more accurately. However, existing models tend to ignore the implicit knowledge in the images and focus only on the visual information in the images, which limits the understanding depth of the image content. The images contain more than just visual objects, some images contain textual information about the scene, and slightly more complex images contain relationships between individual visual objects. Firstly, this paper proposes a model using image description for feature enhancement. This model encodes images and their descriptions separately based on the question-guided coattention mechanism. This mechanism increases the feature representation of the model, enhancing the model’s ability for reasoning. In addition, this paper improves the bottom-up attention model by obtaining two image region features. After obtaining the two visual features and the spatial position information corresponding to each feature, concatenating the two features as the final image feature can better represent an image. Finally, the obtained spatial position information is processed to enable the model to perceive the size and relative position of each object in the image. Our best single model delivers a 74.16% overall accuracy on the VQA 2.0 dataset, our model even outperforms some multi-modal pre-training models with fewer images and a shorter time. [ABSTRACT FROM AUTHOR]