Search

Existing retinex-based low-light image enhancement strategies focus heavily on crafting complex networks for Retinex decomposition but often result in imprecise estimations. To overcome the limitations of previous methods, we introduce a straightforward yet effective strategy for Retinex decomposition, dividing images into colormaps and graymaps as new estimations for reflectance and illumination maps. The enhancement of these maps is separately conducted using a diffusion model for improved restoration. Furthermore, we address the dual challenge of perturbation removal and brightness adjustment in illumination maps by incorporating brightness guidance. This guidance aids in precisely adjusting the brightness while eliminating disturbances, ensuring a more effective enhancement process. Extensive quantitative and qualitative experimental analyses demonstrate that our proposed method improves the performance by approximately 4.4 % on the LOL dataset compared to other state-of-the-art diffusion-based methods, while also validating the model's generalizability across multiple real-world datasets. [ABSTRACT FROM AUTHOR]

Since the fully convolutional network has achieved great success in semantic segmentation, lots of works have been proposed to extract discriminative pixel representations. However, the authors observe that existing methods still suffer from two typical challenges: (i) The intra‐class feature variation between different scenes may be large, leading to the difficulty in maintaining the consistency between same‐class pixels from different scenes; (ii) The inter‐class feature distinction in the same scene could be small, resulting in the limited performance to distinguish different classes in each scene. The authors first rethink semantic segmentation from a perspective of similarity between pixels and class centers. Each weight vector of the segmentation head represents its corresponding semantic class in the whole dataset, which can be regarded as the embedding of the class center. Thus, the pixel‐wise classification amounts to computing similarity in the final feature space between pixels and the class centers. Under this novel view, the authors propose a Class Center Similarity (CCS) layer to address the above‐mentioned challenges by generating adaptive class centers conditioned on each scenes and supervising the similarities between class centers. The CCS layer utilises the Adaptive Class Center Module to generate class centers conditioned on each scene, which adapt the large intra‐class variation between different scenes. Specially designed Class Distance Loss (CD Loss) is introduced to control both inter‐class and intra‐class distances based on the predicted center‐to‐center and pixel‐to‐center similarity. Finally, the CCS layer outputs the processed pixel‐to‐center similarity as the segmentation prediction. Extensive experiments demonstrate that our model performs favourably against the state‐of‐the‐art methods. [ABSTRACT FROM AUTHOR]

Visual Question Answering (VQA) is a multimodal task, which requires understanding the information in the natural language questions and paying attention to the useful information in the images. So far, the solution of VQA tasks can be divided into grid-based methods and bottom-up-based methods. The grid-based method directly extracts the semantic features of the image by leveraging the convolution neural network (CNN), so it has a praiseworthy computational efficiency, but the global convolution feature ignores the information of the key area and causes the performance bottleneck. The bottom-up-based method needs to detect potentially problem-related objects by using some object detection frameworks, e.g. Faster RCNN, so it has better performance, but the computational efficiency is reduced due to the computation of Region Proposal Network (RPN) and Non-Maximum Suppression (NMS). Based on the aforementioned reasons, we propose a fully transformer-based network (FTN) that can maintain a balance between computational efficiency and accuracy, which can be trained end-to-end and consists of three components: question module, image module, and fusion module. Meanwhile, the image module and the question module are visualized to explore the operating rules of the transformer. The experiment results demonstrate that the FTN can focus on key information and objects in the question module and the image module, and our single model has reached 69.01% accuracy on the VQA2.0 dataset, which is superior to the grid-based methods. Although FTN fails to surpass a few state-of-the-art bottom-up-based methods, the FTN has obvious advantages in computational efficiency. The code will be released at https://github.com/weixiang-xu/FTN-VQA.git. [ABSTRACT FROM AUTHOR]