Search

Bridging visual and textual representations plays a central role in delving into multimedia data understanding. The main challenge arises from that images and texts exist in heterogeneous spaces, leading to the difficulty to preserve the semantic consistency between both modalities. To narrow the modality gap, most recent methods resort to extra object detectors or parsers to obtain the hierarchical representations. In this work, we address this problem by introducing our Multi-Task Hierarchical Convolutional Neural Network (MT-HCN). It is characterized by mining the hierarchical semantic information without the aid of any extra supervisions. Firstly, from the perspective of representing architecture, we leverage the intrinsic hierarchical structure of Convolutional Neural Networks (CNNs) to decompose the representations of both modalities into two semantically complementary levels, i.e. , exterior representations and concept representations. The former focuses on discovering the fine-grained low-level associations between both modalities, meanwhile the latter underlines capturing more high-level abstract semantics. Specifically, we present a Self-Supervised Clustering (SSC) loss to preserve more fine-grained semantic clues in exterior representations. It is constituted on the basis of viewing multiple image/text pairs with similar exterior as a category. In addition, a novel harmonious bidirectional triplet ranking (HBTR) loss is proposed, which mitigate the adverse effects brought about by the biased and noisy negative samples. Besides hardest negatives, it also imposes the constraints on the distance between the positive pairs and the centroid of negative pairs. Extensive experimental results on two popular cross-modal retrieval benchmarks demonstrate our proposed MT-HCN can achieve the competitive results compared with the state-of-the-art methods. • This paper proposes a novel Multi-Task Hierarchical Convolutional Network (MT-HCN) for visual-semantic cross-modal retrieval, which is characterized by adopting classification task to improve the hierarchical multi-modal representation learning. • This paper proposes a novel Self-Supervision Clustering (SSC) loss to learn the exterior representations that fully exploits low-level fine-grained correlation for associating images and texts. • This paper presents an effective bidirectional ranking loss, namely Harmonious Bidirectional Ranking (HBR) for cross-modal correlation preserving. It not only efficiently assists us to seek out more representative hard negative samples, but also leverages the category center of negatives to enhance the robustness of cross-modal representations. • Extensive experiments on two benchmark datasets validate the superiority of our proposed model in comparison to the state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

In pattern recognition tasks, the information from system input is modeled through a series of nonlinear operations, which include but not limited to feature extraction, regression, and classification. Both theoretically and practically, these operations are inevitably subject to internal modeling error and external disturbance, resulting at a performance challenge. Those state-of-the-art methods, e.g. Convolutional Neural Network and Transformer, still display significant instabilities and failures under practical applications, so comes a lack of generalization. Consequently, the more robust pattern recognition methods and related theories still merit a further study. This paper firstly reviews those state-of-the-art technologies in the field. The bottleneck of performances in those latest researches is associated with a lack of disturbance estimation and corresponding compensation. Therefore, the implications of disturbance rejection in pattern recognition field are further discussed from a control point of view. Then, the open problems are summarized. Ultimately, a discussion of the potential solutions, which is related to the application of compensation on features, is given to highlight the future study. Through the systematic review in this paper, the disturbance rejection in pattern recognition is developed into a control problem. Hopefully, more effective control technologies for the compensation on features can be used to improve the robustness of pattern recognition theoretically and practically. • The pattern recognition process is inevitably subject to internal modeling error and external disturbance. • The shortcomings of existing disturbance rejection technologies are summarized. • The bottleneck of performances in those latest researches is associated with a lack of disturbance estimation and corresponding compensation. • The open problems and the potential solutions are summarized to discuss the future study in an innovative control point of view, i.e. a compensation on features. [ABSTRACT FROM AUTHOR]

Deep convolutional neural networks (CNNs) have been widely applied to cataract recognition tasks and have achieved promising results. However, most existing methods focused on designing data-driven CNN architectures, and failed to exploit asymmetric opacity distribution prior of cataract, which is significant for cataract diagnosis. To this end, this paper proposes a regional context-based recalibration (RCR) module, which fully leverages the clinical prior to recalibrate the feature maps with regional pooling, region-based context integration, and integrated context fusion. We stack these RCR modules to form an RCRNet based on anterior segment optical coherence tomography (AS-OCT) images for cataract recognition. Experiments on the AS-OCT-NC2 dataset and two publicly available medical datasets demonstrate that RCRNet achieves a better trade-off between performance and efficiency than state-of-the-art channel attention-based networks. We also explain the inherent behavior of RCRNet with the aid of the visual analysis. In addition, this paper is the first to study the effects of two performance evaluation methods on AS-OCT image-based cataract classification results: the single-image level and the single-eye level, suggesting that adopting the single-eye level to evaluate cataract classification performance according to clinical diagnosis requirement. • We design a regional context-based recalibration module by leveraging the prior of cataract. • We construct an RCRNet for cataract recognition based on AS-OCT images. • We test the effects of two performance evaluation methods on cataract classification performance. [ABSTRACT FROM AUTHOR]

Deep clustering has shown its promising capability in joint representation learning and clustering via deep neural networks. Despite the significant progress, the existing deep clustering works mostly utilize some distribution-based clustering loss, lacking the ability to unify representation learning and multi-scale structure learning. To address this, this paper presents a new deep clustering approach termed I mage c luster i ng with c ontrastive le arning and multi-scale G raph C onvolutional N etworks (IcicleGCN), which bridges the gap between convolutional neural network (CNN) and graph convolutional network (GCN) as well as the gap between contrastive learning and multi-scale structure learning for the deep clustering task. Our framework consists of four main modules, namely, the CNN-based backbone, the Instance Similarity Module (ISM), the Joint Cluster Structure Learning and Instance reconstruction Module (JC-SLIM), and the Multi-scale GCN module (M-GCN). Specifically, the backbone network with two weight-sharing views is utilized to learn the representations for the two augmented samples (from each image). The learned representations are then fed to ISM and JC-SLIM for joint instance-level and cluster-level contrastive learning, respectively, during which an auto-encoder in JC-SLIM is also pretrained to serve as a bridge to the M-GCN module. Further, to enforce multi-scale neighborhood structure learning, two streams of GCNs and the auto-encoder are simultaneously trained via (i) the layer-wise interaction with representation fusion and (ii) the joint self-adaptive learning. Experiments on multiple image datasets demonstrate the superior clustering performance of IcicleGCN over the state-of-the-art. The code is available at https://github.com/xuyuankun631/IcicleGCN. • This paper for the first time enables multi-scale structure learning for image clustering via GCNs. • Multi-scale structure learning and two levels of contrastive learning are jointly enforced. • A novel deep image clustering approach termed IcicleGCN is proposed. • Extensive experimental results confirm the superiority of IcicleGCN over the state-of-the-art. [ABSTRACT FROM AUTHOR]

Recently, new sensors with active pixels were brought to market. These sensors export local variations of light intensity in the form of asynchronous events with low latency. Since the data output format is a stream of addressable events and not a complete image of light intensities, new algorithms are required for known problems in the field of Computer Vision, such as segmentation, VO, SLAM, object, and scene recognition. There are some proposed methodologies for object recognition using conventional methods, convolutional neural networks, and third-generation neural networks based on spikes. However, convolutional neural networks and spike neural networks require specific hardware for processing, hard to miniaturize. Also, several traditional Computer Vision operators and feature descriptors have been neglected in the context of event sensors and could contribute to lighter methodologies in object recognition. This paper proposes an algorithm for local binary pattern extraction in sparse structures, typically found in this context. This paper also proposes two methodologies using local binary patterns to captures with event-based sensors for object recognition. The first methodology exploits the known motion performed by the sensor, while the second is motion agnostic. It is demonstrated experimentally that the LBP operator is a fast and light alternative that enables variable reduction using PCA in some cases. The experiments also show that it is possible to reduce the final feature vector for classification by up to 99 , 73 % when compared to conventional methods considered state-of-the-art while maintaining comparable accuracy. • Sparse-LBP extraction algorithm. • Methodologies for memory-efficient object recognition with event-based sensors. • Surface of active events representation using hash tables. [ABSTRACT FROM AUTHOR]

• LPSF module is designed to improve the gas features. • Collaborative attention is proposed for local and global gas features. • The feature compensation mechanism of CNN and Transformer is designed. • PSCFormer is proposed for gas recognition. Based on their powerful feature extraction capability, a convolutional neural network (CNN) has been gradually applied to gas identification in the electronic nose (e-nose) system. The responses of different intensities in the e-nose system are significantly correlated, and CNN extracts the local gas features by convolution while ignoring their global correlation. Transformer combines different responses and obtains the correlation between global features by self-attention. This paper proposes a lightweight hybrid network called Peak Search-based Convolutional Transformers (PSCFormer). First, combining the data characteristics of gas information, the Local Peak Search and Feature Fusion (LPSF) module is proposed to focus on the key gas features. Second, Transformer Encoder (TE) is proposed to obtain the global correlation between global features, and the parallel Convolution Encoder (CE) is proposed to capture the local dependence. Finally, a reasonable feature complementation mechanism is presented, and the preference of TE is alleviated for the slow-down response while solving the receptive field limitation of CE. This paper has evaluated three different datasets to validate the effectiveness of PSCFormer, all of which show stable and excellent performance with a good tradeoff between efficiency and complexity. The results prove that PSCFormer is an efficient and lightweight gas identification network, which provides a method to promote the engineering application of the e-nose system. [ABSTRACT FROM AUTHOR]

• This paper explores the potential of spatial feature dimensions to seek an effective feature representation for point clouds. • The dimension expansion of MKConv enables spatially correlated feature representation and discrete convolutions for point cloud data without information loss. • For stable and effective learning, kernel weight normalization and multidimensional attention are proposed. • The effectiveness of multidimensional feature representation is demonstrated by the superior performance on three different point cloud processing tasks including object classification, object part segmentation, and scene semantic segmentation. Despite the remarkable success of deep learning, an optimal convolution operation on point clouds remains elusive owing to their irregular data structure. Existing methods mainly focus on designing an effective continuous kernel function that can handle an arbitrary point in continuous space. Various approaches exhibiting high performance have been proposed, but we observe that the standard pointwise feature is represented by 1D channels and can become more informative when its representation involves additional spatial feature dimensions. In this paper, we present Multidimensional Kernel Convolution (MKConv), a novel convolution operator that learns to transform the point feature representation from a vector to a multidimensional matrix. Unlike standard point convolution, MKConv proceeds via two steps. (i) It first activates the spatial dimensions of local feature representation by exploiting multidimensional kernel weights. These spatially expanded features can represent their embedded information through spatial correlation as well as channel correlation in feature space, carrying more detailed local structure information. (ii) Then, discrete convolutions are applied to the multidimensional features which can be regarded as a grid-structured matrix. In this way, we can utilize the discrete convolutions for point cloud data without voxelization that suffers from information loss. Furthermore, we propose a spatial attention module, Multidimensional Local Attention (MLA), to provide comprehensive structure awareness within the local point set by reweighting the spatial feature dimensions. We demonstrate that MKConv has excellent applicability to point cloud processing tasks including object classification, object part segmentation, and scene semantic segmentation with superior results. [ABSTRACT FROM AUTHOR]

Deep learning techniques have been widely used for two-dimensional (2D) and three-dimensional (3D) computer vision problems, such as object detection, super-resolution (SR) and classification to name a few. Radar images suffer from poor resolution as compared to optical images, hence developing a high-accuracy model to solve computer vision problems, such as a classifier, is a challenge. This is because of the lack of high-frequency details in the input images which makes it difficult for the classifier model to generate accurate predictions. Ways of addressing this challenge include training the learning model with a large dataset or using a more complicated model, such as deeper layer architecture. However, employing such approaches might result in the overfitting of the model, where the model might not generalize well on previously unseen data. Also, generating a large dataset for training the model is a challenging task, especially in the case of radar images. An alternate solution for achieving high accuracy in radar classification problems is provided in this paper wherein a CNN-enabled super-resolution (SR) model is integrated with the classifier model. The SR model is trained to generate high-resolution (HR) millimeter-wave (mmW) images from any input low-resolution (LR) mmW images. These resolved images from the SR model will be used by the classifier model to classify the input images into appropriate classes, consisting of threat and non-threat objects. The training data for the dual CNN layers are generated using a numerical model of a near-field coded-aperture computational imaging (CI) system. This trained dual CNN model is tested with simulated data generated from the CI numerical model wherein a high classification accuracy of 95% and a fast inference time of 0.193 s are obtained, making it suitable for real-time automated threat classification applications. For fair comparison, the developed CNN model is also validated with experimentally generated reconstruction data, in which case, a classification accuracy of 94% is obtained. • An integrated convolutional neural network is developed which jointly performs super-resolution and classification tasks. The super-resolution part enhances the resolution of millimeter-wave input images and the classifier part predicts a class for the enhanced image. • The super-resolution part of the model enhances both the cross-range and range resolutions of the input images. In addition, the model uses both the real and imaginary parts of the reconstruction data to perform the super-resolution task. • The super-resolution and the classifier models are trained on synthesized data instead of experimentally generated data. To offer a realistic approach, additive Gaussian noise is incorporated into the synthesized data. • Validation of the integrated model with both synthesized and experimental data shows a real-time accurate super-resolution and multi-label classification performance. [ABSTRACT FROM AUTHOR]

Image guided depth completion aims to predict a dense depth map from sparse depth measurements and the corresponding single color image. However, most state-of-the-art methods only rely on convolutional neural network (CNN) or transformer. In this paper, we propose a transformer-CNN parallel network (TCPNet) to integrate the advantages of CNN in local detail recovery and transformer in long-range semantic modeling. Specifically, our CNN branch adopts dense connection to strengthen feature propagation. Since the common transformer computes self-attention based on all the tokens in the window, no matter if they are relevant or not, this will inevitably introduce interferences and noises. To improve the self-attention accuracy, we propose a correlation-based transformer to only allow nearest neighbor tokens to participate in the self-attention computation. We also design a multi-scale conditional random field (CRF) module to implement multi-scale high-dimensional filtering for depth refinement. The comprehensive experimental results on KITTI and NYUv2 demonstrate that our method outperforms the state-of-the-art methods. • We propose a transformer-CNN parallel network by exploiting local details and global contexts. • We design a correlation-based transformer branch to compute attention only with the top-k relevant tokens. • We propose a multi-scale conditional random field module by adaptively assembling multiple dilated CRF for depth refinement. [ABSTRACT FROM AUTHOR]

In image denoising, deep convolutional neural networks (CNNs) can obtain favorable performance on removing spatially invariant noise. However, many of these networks cannot perform well on removing the real noise (i.e. spatially variant noise) that is generated during image acquisition or transmission, which severely impedes their application in practical image denoising tasks. In this paper, we propose a novel Dual-branch Residual Attention Network (DRANet) for image denoising, which has both the merits of a wide model architecture and the attention-guided feature learning. The proposed DRANet includes two different parallel branches, which can capture complementary features to enhance the learning ability of the model. We designed a new residual attention block (RAB) and a novel hybrid dilated residual attention block (HDRAB) for the upper and lower branches, respectively. The RAB and HDRAB can capture rich local features through multiple skip connections between different convolutional layers, and the unimportant features can be dropped. Meanwhile, the long skip connections in each branch and the global feature fusion between the two parallel branches can effectively capture the global features as well. Extensive experiments demonstrate that compared with other state-of-the-art denoising methods, our DRANet can produce competitive denoising performance both on the synthetic and real-world noise removal. The code for DRANet is accessible at https://github.com/WenCongWu/DRANet. • A novel dual residual attention network (DRANet) is designed for image blind denoising, which is effective for both the synthetic noise and real-world noise. • Two types of residual attention blocks (RDAB and HDRAB) are proposed for the two branches of our model respectively, which can extract complementary features and implement feature filtering. • Extensive experiments have verified that the DRANet can obtain competitive denoising results compared to many other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Standard deep learning models such as convolutional neural networks (CNNs) lack the ability of generalizing to domains which have not been seen during training. This problem is mainly due to the common but often wrong assumption of such models that the source and target data come from the same i.i.d. distribution. Recently, Vision Transformers (ViTs) have shown outstanding performance for a broad range of computer vision tasks. However, very few studies have investigated their ability to generalize to new domains. This paper presents a first Token-level Feature Stylization (TFS-ViT) approach for domain generalization, which improves the performance of ViTs to unseen data by synthesizing new domains. Our approach transforms token features by mixing the normalization statistics of images from different domains. We further improve this approach with a novel strategy for attention-aware stylization, which uses the attention maps of class (CLS) tokens to compute and mix normalization statistics of tokens corresponding to different image regions. The proposed method is flexible to the choice of backbone model and can be easily applied to any ViT-based architecture with a negligible increase in computational complexity. Comprehensive experiments show that our approach is able to achieve state-of-the-art performance on five challenging benchmarks for domain generalization, and demonstrate its ability to deal with different types of domain shifts. The implementation is available at this repository. • A new token-level stylization method for Domain Generalization is introduced. • We further enhance the method with a novel attention-driven styling strategy. • Our method is flexible and easily adaptable to any ViT, adding minimal complexity. • Our method shows superior performance in most cases on five challenging datasets. [ABSTRACT FROM AUTHOR]

• New framework to accurately and most efficiently match real-world contactless and cross-sensor palmprint images. • Outperforming results over state-of-the-art methods, using most challenging match protocols, on several publicly accessible datasets. • Significantly faster than previous approaches introduced in the contactless palmprint literature while achieving much higher match accuracy. • Rigorous cross-database and cross-sensor performance evaluation, using palm images from mobile phones, and ROC from over 35 million match scores. Contactless palmprint identification offers significantly improved hygiene and user convenience, making it highly attractive for a range of civilian applications, especially during the current pandemic. However, the accurate recognition of contactless palmprint images can be highly challenging, attributed to the significant variations in the intra-class similarity and limitations of conventional palmprint feature descriptors under involuntary or contactless imaging variations. State-of-the-art completely contactless palmprint matching algorithms in the literature cannot adequately address these challenges and are not sufficiently accurate and fast enough for such real-world applications. This paper proposes a novel approach that adaptively locates the local palmprint regions with high similarities between their corresponding feature representations or templates to address these challenges. We consider spatial localization of such highly similar feature representations from multiple local regions and consolidate them to generate a more reliable match score. This paper presents reproducible and comparative experimental results, using within-database, cross-database, and cross-sensor performance evaluation, on four publicly available contactless palmprint datasets, including a sizeable contactless palmprint database from 600 different subjects. The proposed method achieves outperforming results compared with three state-of-the-art deep learning-based methods and five widely used conventional methods. In addition, the proposed method is also significantly faster than all state-of-the-art baseline methods. [ABSTRACT FROM AUTHOR]

Benefiting from the development of deep convolutional neural networks, CNN-based single-image super-resolution methods have achieved remarkable reconstruction results. However, the limited perceptual field of the convolutional kernel and the use of static weights in the inference process limit the performance of CNN-based methods. Recently, a few vision transformer-based image super-resolution methods have achieved excellent performance compared to CNN-based methods. These methods contain many parameters and require vast amounts of GPU memory for training. In this paper, we propose a spatial shuffle multi-head self-attention for single-image super-resolution that can significantly model long-range pixel dependencies without additional computational consumption. A local perception module is also proposed to combine convolutional neural networks' local connectivity and translational invariance. Reconstruction results on five popular benchmarks show that the proposed method outperforms existing methods in both reconstruction accuracy and visual performance. The proposed method matches the performance of transformed-based methods but requires an inferior number of transformer blocks, which reduces the number of parameters by 40%, GPU memory by 30%, and inference time by 30% compared to transformer-based methods. • We used attribution analysis to find that some transformer based SR methods can only utilize limited spatial range information during the reconstruction process. • To address this, we introduce Spatial Shuffle Multi-Head Self-attention (SS-MSA) for efficient global pixel dependency modeling and a local perceptual unit to enhance local feature information. • Our method surpasses existing approaches in reconstruction accuracy and visual performance across five benchmarks. Moreover, it reduces parameters by 40%, GPU memory by 30%, and inference time by 30% compared to transformer-based methods. [ABSTRACT FROM AUTHOR]

In this paper, we present H-CapsNet, a capsule network for hierarchical image classification. Our network makes use of the natural capacity of CapsNets (capsule networks) to capture hierarchical relationships. Thus, our network is such that each multi-layer capsule network accounts for each of the class hierarchies using dedicated capsules. Further, we make use of a modified hinge loss that enforces consistency amongst the hierarchies involved. We also present a strategy to dynamically adjust the training parameters to achieve a better balance between the class hierarchies under consideration. We have performed experiments using several widely available datasets and compared them against several alternatives. In our experiments, H-CapsNet delivers a margin of improvement over competing hierarchical classification networks elsewhere in the literature. • Propose H-CapsNet for hierarchical classification with dedicated capsules per level. • Enforce hierarchical consistency with label-tree guided modified hinge-loss. • Adjust training parameters dynamically to balance hierarchical level contributions. • Deliver a margin of improvement over alternatives on benchmark datasets. [ABSTRACT FROM AUTHOR]

The challenge of image generation has been effectively modeled as a problem of structure priors or transformation. However, existing models have unsatisfactory performance in understanding the global input image structures because of particular inherent features (for example, local inductive prior). Recent studies have shown that self-attention is an efficient modeling technique for image completion problems. In this paper, we propose a new architecture that relies on Distance-based Weighted Transformer (DWT) to better understand the relationships between an image's components. In our model, we leverage the strengths of both Convolutional Neural Networks (CNNs) and DWT blocks to enhance the image completion process. Specifically, CNNs are used to augment the local texture information of coarse priors and DWT blocks are used to recover certain coarse textures and coherent visual structures. Unlike current approaches that generally use CNNs to create feature maps, we use the DWT to encode global dependencies and compute distance-based weighted feature maps, which substantially minimizes the problem of visual ambiguities. Meanwhile, to better produce repeated textures, we introduce Residual Fast Fourier Convolution (Res-FFC) blocks to combine the encoder's skip features with the coarse features provided by our generator. Furthermore, a simple yet effective technique is proposed to normalize the non-zero values of convolutions, and fine-tune the network layers for regularization of the gradient norms to provide an efficient training stabilizer. Extensive quantitative and qualitative experiments on three challenging datasets demonstrate the superiority of our proposed model compared to existing approaches. • Proposed a generative model for coherent image completion. • Proposed Distance-based Weighted Transformer to encode global dependencies. • Proposed a norm-regularization method to stabilize training. [ABSTRACT FROM AUTHOR]

Benefiting from deep learning, Salient Object Detection (SOD) has made much progress. However, most existing methods adopt the same strategy to extract salient cues from different feature levels without fully considering their differences in the feature extraction stage and/or suffer from the accumulation of noise and dilution of spatial details in the feature fusion stage. These two problems hinder the further improvement in performance. In this paper, we propose an effective SOD model, PiNet, which can address the above problems via two novel mechanisms in the network: level-specific feature extraction and progressive refinement of saliency. We have designed the customized feature extraction components for each level of features—enabling us to extract better saliency cues from multi-level features. The saliency feature refinement in the branches follows a coarse-to-fine process, where the refined features progressively contain more location cues, internal and boundary details. Through short connections, the extracted saliency cues in different branches are selectively transmitted and integrated, which well mitigates the accumulation of noisy information and the dilution of detailed information. By using four different backbones, we verify our model has good adaptability and can make accurate saliency predictions under different pretrained models. Extensive experiments on five public datasets demonstrate that PiNet outperforms 19 state-of-the-art (SOTA) methods in SOD, with its small model size (56.1 MB) and fast inference speed (47 FPS). • An effective model, PiNet, is proposed for SOD with two novel mechanisms. • A Smoothed Feature Attention Module is designed to consider the location of salient objects. • A Feature Interaction Module is designed to guarantee the consistency in the interior of salient objects. • A Progressive Fusion Module is designed to effectively aggregate edge cues and saliency features. • PiNet outperforms 19 state-of-the-art methods and reaches a real-time speed for SOD. [ABSTRACT FROM AUTHOR]

Deep learning (DL)-based models especially Convolutional Neural Network (CNN) models have recently achieved great success in medical image classifications. It is usually time-consuming and labor-intensive to train a practical classification model due to the requirement of large data volume. Since medical images are more difficult to acquire and label for model training, many scholars have applied transfer learning by pre-training a model on a larger dataset and then fine-tuning it on the target dataset to obtain better classification results. However, such approach, relying on individual expertise to select related datasets, is subjective and inconsistent in performance. In this paper, we propose a simple yet effective method for measuring the transferability between different datasets, and build a Dataset Map (DataMap) that can be used as a tool to find the most relevant datasets for transfer learning on target dataset. Recent studies show that the convolutional kernels in CNN models have different function roles. Therefore, we adopt the similarity between the convolution kernels to measure the transferability between datasets. Firstly, the gradient attribution is adopted to attribute the task related convolution kernels from last few convolution layers of the same pre-trained model architecture trained with different datasets. Then, the similarity between attributed convolutional kernels is calculated to denote the transferability between different datasets. Finally, we build a DataMap with 20 medical image datasets. Extensive experimental tests on 3 mainstream CNN architectures show that the proposed method can effectively measure the transferability between different datasets. With the guidance of the DataMap, the transfer learning can achieve the best performance on various training tasks, and the accuracy of the CNN classifier can be improved by 1% to 5% through pre-training. • The first DataMap used to instruct transfer learning for medical image classificaton. • Adopt the similarity between the convolution kernels to measure the transferability. • With the guidance of DataMap, transfer learning can achieve the best performance. [ABSTRACT FROM AUTHOR]

Graph convolutional neural networks (GCNs) have demonstrated powerful representing ability of irregular data, e.g., skeletal data and graph-structured data, providing the effective mechanism to fuse the neighbor nodes. However, inheriting from the deep learning, GCN also lacks interpretability, which hinders its application to scenarios that have high demand for transparency. Although, there have been many efforts on the interpretability of deep learning, they mainly concentrate on i.i.d data that is hard to be deployed to GCNs, which involve not only the node feature, but also the graph structure. There are few works that attempt to explain it with post-hoc manner, which can be biased, resulting in mis-representation of the true explanation. Therefore, in this paper, we propose a framework, namely ExpFiGCN, that reveals explainability of the GCNs from the perspective of graph structure and mathematical analysis. Specifically, ExpFiGCN can find the most intrinsically relevant node to the central node and obtain the informative and discriminative signals while performing denoising. For the graph structure, we find K -nearest nodes; for the mathematical analysis, every channel of a node and its neighborhoods contribute dynamically to the final channel signal, which can capture the inherent difference of different channels and neighbor nodes. Meanwhile, it can enhance the representation ability of nodes and ameliorate the over-smoothing problem. On the other hand, our model can dynamically adjust the importance of neighborhoods to the central vertex. We empirically validate the effectiveness of the proposed framework ExpFiGCN on various benchmark datasets. Experimental results show that our method achieves substantial improvements and outperforms the state-of-the-art performance strikingly. • We propose ExpFiGCN explaining GCNs by structure of graph and mathematical analysis. • Our framework effectively tackles the unequal importance of information. • Our framework ExpFiGCN implementation is simple and efficient. • Our method achieves promising results and outperform SOTA methods significantly. [ABSTRACT FROM AUTHOR]

Texture representation is a challenging problem due to the complex underlying physics of texture as well as the variations caused by changes in viewpoint. Recent progress in texture analysis has been made by the power of convolutional neural networks (CNNs) in feature learning. However, most current methods aggregate the features from the last convolutional layer of the CNN to obtain a global feature vector, which fails to leverage shallow low-level visual cues and cross-layer feature patterns, limiting their performance. In this paper, we propose to trace the features generated along the convolutional layers via a histogram of local 3D invariant binary patterns, called deep tracing patterns. This leads to a highly discriminative yet robust global feature representation module. Building such a module into a CNN backbone, we develop an effective approach for texture recognition. Extensive experiments on six benchmark datasets show that the proposed approach provides a discriminative and robust texture descriptor, with state-of-the-art performance achieved. • Deep feature representation from discriminative cross-layer statistics in a CNN. • A rotation-invariant scheme for encoding cross-layer feature patterns. • A deep CNN with excellent performance in texture classification and retrieval. [ABSTRACT FROM AUTHOR]

In this paper, we explore the spatial redundancy in continuous sign language recognition (CSLR), aiming to improve its efficiency. Despite recent advances in accuracy in CSLR, state-of-the-art CSLR methods typically require large amounts of computations and memory occupation, which are not friendly towards fast inference under limited computation/memory budgets. Based on a simple observation that not all frames are equally important for CSLR, we propose AdaSize to handle this problem by modeling the frame resolution decision as an end-to-end learnable task to save unnecessary computations. Specifically, a lightweight 2D convolutional neural network (CNN) is first used to quickly browse input frames under a low resolution (e.g., 112 × 112). These extracted coarse and cheap features are sent into a recurrent policy network to dynamically determine the desired resolution for each frame. Once the optimal resolution for each frame is decided, frames with different resolutions are fed into the following backbones to extract representative features. Finally, these features pass through a sequence of temporal modules and a classifier to predict sentences. Extensive experiments on four large-scale datasets, including PHOENIX14, PHOENIX14-T, CSL-Daily and CSL, demonstrate the effectiveness of AdaSize. AdaSize could consistently achieve comparable accuracy with state-of-the-art CSLR methods, with only 0.38 × computations, 0.41 × memory usage and 1.25 × throughput. Comparisons with commonly-used lightweight backbones and other efficient methods verify the superiority of AdaSize under similar computational/memory budgets. We finally plot the frame resolution decisions for AdaSize, hoping to provide insightful analysis of the inherent spatial redundancy in videos. • Sign language is the main communication tool of the hearing-impaired people in their daily life. • Current sign language systems consume large computational costs and own high inference latency. • A novel adaptive method to improve network efficiency by switching among various resolution. • Both online and offline ablations on the generalizability of the proposed method. [ABSTRACT FROM AUTHOR]

Signature verification is a frequently-used forensics technology in numerous safety-critical situations. Although convolutional neural networks (CNNs) have made significant advancements in the field of signature verification, their reliance on local neighborhood operations poses limitations in capturing the global contextual relationships among signature strokes. To overcome this weakness, in this paper, we propose a novel holistic-part unified model named TransOSV based on the vision transformer framework to solve offline signature verification problem. The signature images are first encoded into patch sequences by the proposed transformer-based holistic encoder to learn the global signature representation. Second, considering the subtle local difference between the genuine signature and forged signature, we design a contrast based part decoder along with a sparsity loss, which are utilized to learn the discriminative part features. With the learned holistic features and part features, the proposed model is optimized by the contrast loss function. To reduce the influence of sample imbalance, we also formulate a new focal contrast loss function. Furthermore, we conduct the proposed model to learn signature representations for writer-dependent signature verification task. The experimental results demonstrate the potential of the proposed TransOSV model for both writer-independent and writer-dependent signature verification tasks, achieving remarkable performance improvements and competitive results on four widely-used offline signature datasets. • It proposes a TransOSV model to verify transformers on signature verification tasks. • It proposes a novel holistic encoder and a contrast based part decoder. • A focal contrast loss is formulated to collaborate with the proposed model. • We employ the model to learn representations for WD signature verification tasks. [ABSTRACT FROM AUTHOR]

Over the last few years, the availability of an increasing data generated from non-Euclidean domains, which are usually represented as graphs with complex relationships, and Graph Neural Networks (GNN) have gained a high interest because of their potential in processing graph-structured data. In particular, there is a strong interest in performing convolution on graphs using an extension of the GNN architecture, generally referred to as Graph Convolutional Neural Networks (ConvGNN). Convolution on graphs has been achieved mainly in two forms: spectral and spatial convolutions. Due to the higher flexibility in exploring and exploiting the graph structure of data, there is recently an increasing interest in investigating the possibilities that the spatial approach can offer. The idea of finding a way to adapt the network behaviour to the inputs they process to maximize the total performances has aroused much interest in the neural networks literature over the years. This paper presents a novel method to adapt the behaviour of a ConvGNN to the input performing spatial convolution on graphs using input-specific filters, which are dynamically generated from nodes feature vectors. The experimental assessment confirms the capabilities of the proposed approach, achieving satisfying results using a low number of filters. • The Convolutional Graph Neural Network parameters are set by another neural network. • Filters are input-specific and dynamically generated exploiting the features of the graph nodes. • The convolution is performed on each graph node with its nearest neighbours. • Graph Neural Network inner behaviour changes according to the input. • Unaltered performances on simpler architectures respect to non-dynamic ones. [ABSTRACT FROM AUTHOR]

• To design and develop a model of counting-based VQA using heuristic-assisted deep learning models for finding the counting answers based on the given simple and complex type questions, which provides the helpful information on the web source or social media application. • To propose the optimal weighted fused feature process, where the image features extracted by visual geometry Group16 (VGG16) and text features retrieved by text convolutional neural network (TCNN) are combined together by determining the optimal weight through novel improved tuna swarm optimization (ITSO) approach. • To frame the novel heuristic algorithm as ITSO, with the new formulation derived for random value that helps in position updating process. It is used to optimize the parameters like epoch in LSTM, hidden neuron of recurrent neural network (RNN) and weights for minimizing the root mean square error (RMSE) value. • To illustrate the novel serial cascaded recurrent neural network with attention mechanism based long short-term memory (SCRAM-LSTM) model, in which the RNN and attention-based LSTM is serially cascaded to estimate the final counting answers. Further, to attain the optimal answers, the hyper-parameters are tuned optimally by improved tuna swarm optimization (ITSO) approach. The counting-based questions play a major part in Visual Question Answering (VQA), the most challenging factor is counting the different objects present in the images. Recently more attention is paid to design a model of count-aided VQA. Based on the questions, the VQA system responds with appropriate answers. Yet, the complex questions are necessitating in the system with answers. The earlier models are still facing the challenging problems of counting the various objects within the images as the models become futile to select the features and lack fine-grained representation. In order to sustain the image representation, this paper proposes a new model for VQA using the heuristic approach of serial cascaded deep learning methods. Initially, the standard data regarding images and text data are gathered and fed to the pre-processing process. Consequently, the feature extraction is done on both the image and the text data. Here, the deep features from images are taken using Visual Geometry Group 16 (VGG16) and the text features are extracted using Text Convolutional Neural Network (TCNN). Then, the optimal weighted fused features are obtained, where the weights used for getting the necessary features are tuned via the Improved Tuna Swarm Optimization (ITSO) algorithm. Finally, the counting answers are retrieved based on the given queries, which is carried out via Serial Cascaded Recurrent Neural Network with Attention Mechanism-based Long Short-Term Memory (SCRAM-LSTM). The performance is examined with divergent metrics compared with conventional models. Hence, the findings reveal that it offers superior performance in estimating the appropriate answers. Therefore, the proposed work is widely used for such potential applications as helping blind or visually impaired people to get information, integrating with image retrieval systems, and also for search engines. Especially, it is utilized for the vision and language systems. [ABSTRACT FROM AUTHOR]

• We propose novel graph classification method for better classification performance. • We develop an edge feature scheme and an add-on layer for discriminative feature learning. • We create corresponding graph datasets based upon ModelNet40, ModelNet10 and ShapeNet Part datasets and will release them to the community. Spectral graph convolutional neural networks (GCNNs) have been producing encouraging results in graph classification tasks. However, most spectral GCNNs utilize fixed graphs when aggregating node features while omitting edge feature learning and failing to get an optimal graph structure. Moreover, many existing graph datasets do not provide initialized edge features, further restraining the ability of learning edge features via spectral GCNNs. In this paper, we try to address this issue by designing an edge feature scheme and an add-on layer between every two stacked graph convolution layers in spectral GCNN. Both are lightweight while effective in filling the gap between edge feature learning and performance enhancement of graph classification. The edge feature scheme makes edge features adapt to node representations at different spectral graph convolution layers. The add-on layer helps adjust the edge features to an optimal graph structure. To test the effectiveness of our method, we take Euclidean positions as initial node features and extract graphs with semantic information from point cloud objects. The node features of our extracted graphs are more scalable for edge feature learning than most existing graph datasets (in one-hot encoded label format). Three new graph datasets are constructed based on ModelNet40, ModelNet10 and ShapeNet Part datasets. Experimental results show that our method outperforms state-of-the-art graph classification methods on the new datasets. Our code and the constructed graph datasets will be released to the community. [ABSTRACT FROM AUTHOR]

• High-dimensional tensor sample formed by crop PolSAR data contains rich structural information. • The "dimension reduction" and "vectorized learning" seriously damaged the structural information. • A novel tensor affine transform neural network (TATN) architecture is built for crop classification. • The proposed TATN model can be trained by error back-propagation strategy. • TATN achieved an accuracy gain of 8.05%, and the number of parameters is only 12.98% of CNN model. Accurate and reliable discrimination of crop categories is a significant data source for agricultural monitoring and food security evaluation research. The convolutional neural network (CNN) model is one of the most popular classifiers for crop discrimination based on polarimetric synthetic aperture radar (PolSAR) data. However, it is better to avoid directly using large amounts of raw features that extracted from PolSAR data for CNN models because of the "dimension disaster" problem caused by multiple periods and various feature extraction schemes. Consequently, an extra feature compression model has to be incorporated to mitigate the "dimension disaster" problem. However, ill coupling of the two models may result in degraded classification performance, thus the combination of two models has to be further optimized. In this paper, we propose a novel single tensor affine transformation network (TATN) for crop classification using multi-temporal PolSAR data, where the input sample of the network is a higher order tensor formed by raw features, and the hidden layers of the network adopt the tensor affine transformation rather than convolution to extract discriminative features for classification. Since the tensor affine transformation preserves the structural information of the original input tensor samples, the TATN is expected to achieve a higher crop classification accuracy. Moreover, the TATN holds less amount of parameters than most of deep learning models, which enables to avoid the extra feature compression procedure. The experimental results validate the merits of our model. [ABSTRACT FROM AUTHOR]

• This paper proposes an improved deep convolutional embedded clustering algorithm using reliable samples. • This paper designs the new deep clustering model structure and corresponding loss function. • In this study, we select reliable samples with pseudo-labels and pass them to the convolutional neural network for training to get a better clustering model. • We conducted experimental tests on four standard data sets and show the better performance compared to the state-of-the-art clustering algorithms. The deep clustering algorithm can learn the latent features of the embedded subspace, and further realize the clustering of samples in the feature space. The existing deep clustering algorithms mostly integrate neural networks and traditional clustering algorithms. However, for sample sets with many noise points, the effect of the clustering remains unsatisfactory. To address this issue, we propose an improved deep convolutional embedded clustering algorithm using reliable samples (IDCEC) in this paper. The algorithm first uses the convolutional autoencoder to extract features and cluster the samples. Then we select reliable samples with pseudo-labels and pass them to the convolutional neural network for training to get a better clustering model. We construct a new loss function for backpropagation training and implement an unsupervised deep clustering method. To verify the performance of the method proposed in this paper, we conducted experimental tests on standard data sets such as MNIST and USPS. Experimental results show that our method has better performance compared to traditional clustering algorithms and the state-of-the-art deep clustering algorithm under four clustering metrics. [ABSTRACT FROM AUTHOR]

• A new network architecture for real-time 3d human pose estimation from RGB images. • Dense re-injection supervision and dense connections at multiple scales. • A new lightweight regression method for 3D human pose estimation. • State-of-the-art results on 3D human pose estimation at more than 120 fps. In this paper we propose a highly scalable convolutional neural networks, end-to-end trainable, for real-time 3D human pose regression from still RGB images. We call this approach Scalable Sequential Pyramid Networks (SSP-Net) as it is trained with refined supervision at multiple scales in a sequential manner. Our network requires a single training procedure and is capable of producing its best predictions at 120 frames per second (FPS), or acceptable predictions at more than 200 FPS when cut at test time. We show that the proposed regression approach is invariant to the size of feature maps, allowing our method to perform multi-resolution intermediate supervisions and reaching results comparable to the state-of-the-art with very low resolution feature maps. We demonstrate the accuracy and the effectiveness of our method by providing extensive experiments on two of the most important publicly available datasets for 3D pose estimation, Human3.6M and MPI-INF-3DHP. Additionally, we provide relevant insights about our decisions on the network architecture and show its flexibility to meet the best precision-speed compromise. [ABSTRACT FROM AUTHOR]

• An end-to-end physical model and image translate fused network is proposed for single image dehazing. The proposed network contains a common feature extraction encoder, a transmission map decoder and an image translation decoder. • An elaborate conditional attention feature fusion block is proposed to aggregate the guidance information of the physical model and the image translation. • The network is supervised with multi-task multi-scale deep supervision mechanism and verified on low-level image dehazing task and high-level semantic segmentation task. The visibility and contrast of images captured in adverse weather such as haze or fog degrade dramatically, which further hinders the accomplishment of high-level computer vision tasks such as object detection and semantic segmentation in these conditions. Many methods have been proposed to solve image dehazing problem by using image translation networks or physical model embedding in CNNs. However, the physical model cannot effectively describe the hazy generation process in complex scenes and estimating the model parameters with only a hazy image is an ill-posed problem. Image translation-based methods may lead to artefacts or colour shifts in the recovered results without the guidance or constraints of physical model information. In this paper, an end-to-end physical model and image translation fused network is proposed to generate realistic haze-free images. Since the transmission map can express the haze distribution in the scene, the proposed method adopts an encoder with a multiscale residual block to extract hazy image features, and two separate decoders to recover a clear image and to estimate the transmission map. The multiscale features of the transmission map and image translation are fused to guide the decode processes with a conditional attention feature fusion block, which is composed of sequential channelwise and spatialwise attention. Moreover, a multitask and multiscale deep supervision mechanism is adopted to enhance the feature fusion and recover more image details. The algorithm can efficiently fuse the physical model information and the hazy image translation to address the problem existent in the methods only based on physical model embedding or direct image translation. Experimental results on the visual quality enhancement of hazy images and semantic segmentation tasks in hazy scenes demonstrate that our model can efficiently recover haze-free images, while performing on par with state-of-the-art methods. [ABSTRACT FROM AUTHOR]

• A novel deep quantization framework using predefined orthonormal vectors as codewords to increase quantization informativeness and reduce codeword redundancy. • A tailored loss function that maximizes discriminability in both the soft quantization and original features in all subspaces. • Highly effective in improving models standard retrieval performance and generalization ability for scalable face image retrieval. • State-of-the-art performance and broad superiority for image retrieval tasks. Existing deep quantization methods provided an efficient solution for large-scale image retrieval. However, the significant intra-class variations, like pose, illumination, and expressions in face images, still pose a challenge. In light of this, face image retrieval requires sufficiently powerful learning metrics, which are absent in current deep quantization works. Moreover, to tackle the growing unseen identities in the query stage, face image retrieval drives more demands regarding model generalization and scalability than general image retrieval tasks. This paper integrates product quantization with orthonormal constraints into an end-to-end deep learning framework to effectively retrieve face images. Specifically, we propose a novel scheme that uses predefined orthonormal vectors as codewords to enhance the quantization informativeness and reduce codewords' redundancy. A tailored loss function maximizes discriminability among identities in each quantization subspace for both the quantized and original features. An entropy-based regularization term is imposed to reduce the quantization error. Experiments are conducted on four commonly-used face datasets under both seen and unseen identity retrieval settings. Our method outperforms all the compared state-of-the-art under both settings. The proposed orthonormal codewords consistently boost both models' standard retrieval performance and generalization ability, demonstrating the superiority of our method for scalable face image retrieval. [ABSTRACT FROM AUTHOR]

• We propose a new surround function to effectively estimate the illumination map. • We propose an Adaptive Retinex Block that can be regarded as an extension of Single Scale Retinex in features space. • We propose a low-exposure denoiser to smooth the image before enhancement. • We proposed a lightweight network that achieves competitive performance against several state-of-the-art methods with less than 150K parameters. Although Convolution Neural Networks (CNNs) have made substantial progress in the low-light image enhancement task, one critical problem of CNNs is the paradox of model complexity and performance. This paper presents a novel SurroundNet that only involves less than 150 K parameters (about 80–98 percent size reduction compared to SOTAs) and achieves very competitive performance. The proposed network comprises several Adaptive Retinex Blocks (ARBlock), which can be viewed as a novel extension of Single Scale Retinex in feature space. The core of our ARBlock is an efficient illumination estimation function called Adaptive Surround Function (ASF). It can be regarded as a general form of surround functions and be implemented by convolution layers. In addition, we also introduce a Low-Exposure Denoiser (LED) to smooth the low-light image before the enhancement. We evaluate the proposed method on two real-world low-light datasets. Experimental results demonstrate the superiority of our submitted SurroundNet in both performance and network parameters against State-of-the-Art low-light image enhancement methods. The code is available at https://github.com/ouc-ocean-group/SurroundNet. [ABSTRACT FROM AUTHOR]

• We propose a novel FETNet which could remove scene text near completely in images. • Our method is formulated in a one stage way and is trained in an end to end manner. • We introduce a novel Flickr ST dataset with multi category careful annotations. • Our FETNet achieves remarkable results on both synthetic and real scene datasets. The scene text removal (STR) task aims to remove text regions and recover the background smoothly in images for private information protection. Most existing STR methods adopt encoder-decoder-based CNNs, with direct copies of the features in the skip connections. However, the encoded features contain both text texture and structure information. The insufficient utilization of text features hampers the performance of background reconstruction in text removal regions. To tackle these problems, we propose a novel Feature Erasing and Transferring (FET) mechanism to reconfigure the encoded features for STR in this paper. In FET, a Feature Erasing Module (FEM) is designed to erase text features. An attention module is responsible for generating the feature similarity guidance. The Feature Transferring Module (FTM) is introduced to transfer the corresponding features in different layers based on the attention guidance. With this mechanism, a one-stage, end-to-end trainable network called FETNet is constructed for scene text removal. In addition, to facilitate research on both scene text removal and segmentation tasks, we introduce a novel dataset, Flickr-ST, with multi-category annotations. A sufficient number of experiments and ablation studies are conducted on the public datasets and Flickr-ST. Our proposed method achieves state-of-the-art performance using most metrics, with remarkably higher quality scene text removal results. [ABSTRACT FROM AUTHOR]

• We proposed a highly efficient online skeleton-based action recognition method. • It reformulates ST-GCN as a Continual Inference Network. • It performs online frame-by-frame predictions in a highly efficient manner. • It achieves up to 26x on-hardware speed-up, and up to 109x FLOP reduction. Graph-based reasoning over skeleton data has emerged as a promising approach for human action recognition. However, the application of prior graph-based methods, which predominantly employ whole temporal sequences as their input, to the setting of online inference entails considerable computational redundancy. In this paper, we tackle this issue by reformulating the Spatio-Temporal Graph Convolutional Neural Network as a Continual Inference Network, which can perform step-by-step predictions in time without repeat frame processing. To evaluate our method, we create a continual version of ST-GCN, Co ST-GCN, alongside two derived methods with different self-attention mechanisms, Co AGCN and Co S-TR. We investigate weight transfer strategies and architectural modifications for inference acceleration, and perform experiments on the NTU RGB+D 60, NTU RGB+D 120, and Kinetics Skeleton 400 datasets. Retaining similar predictive accuracy, we observe up to 109 × reduction in time complexity, on-hardware accelerations of 26 × , and reductions in maximum allocated memory of 52% during online inference. [ABSTRACT FROM AUTHOR]

• This paper proposes a sparse attention block (SA block) to capture long-range dependencies in an efficient way. • SA block introduces the additional cost of < 2 % GPU memory and computation of the conventional non-local block. • SA block can be easily plugged into various object detection frameworks. • SA block boosts the detection accuracy by more than 1% on COCO with slight computation and memory addition. It is well recognized that the contextual information of surrounding objects is beneficial for object detection. Such contextual information can often be obtained from long-range dependencies. This paper proposes a sparse attention block to capture long-range dependencies in an efficient way. Unlike the conventional non-local block, which generates a dense attention map to characterize the dependency between any two positions of the input feature map, our sparse attention block samples the most representative positions for contextual information aggregation. After searching for local peaks in a heat map of the given input feature map, it adaptively selects a sparse set of positions to represent the relationship between query and key elements. With the obtained sparse positions, our sparse attention block can well model long-range dependencies, and greatly improve the object detection performance at the additional cost of < 2% GPU memory and computation of the conventional non-local block. This sparse attention block can be easily plugged into various object detection frameworks, such as Faster R-CNN, RetinaNet and Mask R-CNN. Experiments on COCO benchmark confirm that our sparse attention block can boost the detection accuracy with significant gains ranging from 1.4% to 1.9% and negligible overhead of computation and memory usage. [ABSTRACT FROM AUTHOR]

• PRNU is a noise pattern extracted from images used to identify the source camera. • Correlation of image and camera reference PRNUs is used for camera identification. • Even slight de-synchronization of PRNU signals can spoil the identification. • This paper presents a novel method to register scaled and rotated PRNU signals. • PRNU registration is obtained using a deep convolutional network. Assessing if an image comes from a specific device is fundamental in many application scenarios. The most promising techniques to solve this problem rely on the Photo Response Non Uniformity (PRNU), a unique trace left during image acquisition. A PRNU fingerprint is computed from several images of a given device, then it is compared with the probe residual noise by means of correlation. However, such a comparison requires that PRNUs are synchronized: even small image transformations can spoil this task. Most of the attempts to solve the registration problem rely on time consuming brute-force search, which is prone to missing detections and false positives. In this paper, the problem is addressed from a computer vision perspective, exploiting recent image registration techniques based on deep learning, and focusing on scaling and rotation transformations. Experiments show that the proposed method is both more accurate and faster than state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

• A directed convolutional neural network model is proposed • Partial dilation-Global erosion generates resistance against information loss • Several recent approaches for the segmentation of graphemes/characters are discussed • The over-traces issue is addressed for the first time in the literature • MMOs can provide real solutions if a convenient strategy is adopted Due to the nature of Arabic handwriting, segmenting words into characters/graphemes is the most difficult and critical task of the recognition system. The present paper proposes an approach to segment handwritten Arabic words into graphemes based on a directed Convolutional Neural Network (CNN) and Mathematical Morphology Operations (MMO). Arabic script is cursive, which means that almost all graphemes are connected via horizontal links; therefore, a technique to remove links will facilitate the segmentation of graphemes. In general, an MMO such as erosion seems suitable for getting the job done, but since Arabic handwriting is difficult, MMOs cause information loss and suffer from many issues such as diacritics and over-traces, which lead to over/under/bad segmentations. To overcome limitations, the present paper addresses these issues in the following order: the over-traces issue is addressed for the first time in the literature; a robust algorithm for diacritics extraction is provided; and finally, the main segmentation algorithm adopts a strategy based on a Partial Dilation (PD)-Global Erosion (GE) technique to combat the information loss issue. The PD phase amplifies important regions, while GE eliminates links between graphemes. The complementarity between PD and GE facilitates the extraction of graphemes and creates resistance against information loss. To properly tackle these difficult problems, this article exploits the robustness of CNNs, so a new directed CNN model is suggested. The idea is to draw the model's attention to certain targeted features, which are selected according to the nature of the problem addressed. The proposed directed CNN is used in all phases of the segmentation process. The experimental results are very encouraging and show that the proposed directed CNN model outperformed basic CNN in many experiments. The results also reveal that the followed strategy improved the ability of MMOs to perform segmentation and to compete with other approaches in this research area. [ABSTRACT FROM AUTHOR]

• Convolutional Neural Networks can be improved in terms of the classification performance and robustness by using variable weight structures. • Analysis of different data processing methods, models' robustness and statistical properties. • Comparative analysis of variable weight convolutional neural networks and other widely used machine learning techniques. • Medical applications to the classification of seizure phases and types. Deep learning techniques have recently achieved impressive results and raised expectations in the domains of medical diagnosis and physiological signal processing. The widely adopted methods include convolutional neural networks (CNNs) and recurrent neural networks (RNNs). However, the existing models possess static connection weights between layers, which might limit the generalization capability and the classification performance of the models as the weights of different layers are fixed after training. Furthermore, to deal with a large amount of data, a neural network with a sufficiently large size is required. This paper proposes the variable weight convolutional neural networks (VWCNNs), which are a type of network structure employing dynamic weights instead of static weights in their convolutional layers and fully-connected layers. VWCNNs are able to adapt to different characteristics of input data and can be viewed as an infinite number of traditional, fixed-weight CNNs. We will show that the proposed VWCNN structure outperforms the conventional CNN in terms of the classification accuracy, generalization capability, and robustness when the inputs are contaminated by noise. In this paper, VWCNNs are applied to the classification of three seizure phases (seizure-free, pre-seizure and seizure) based on measured electroencephalography (EEG) data. VWCNNs achieve 100% test accuracy and show strong robustness in the classification of the three seizure phases, and thus show the potential to be a useful classification tool for medical diagnosis. Furthermore, the classification of seven types of seizures is investigated in this paper using the world's largest open source database of seizure recordings, TUH EEG seizure corpus. Comparisons with conventional CNNs, RNN, MobileNet, ResNet, DenseNet and traditional machine learning methods including random forest, decision tree, support vector machine, K-nearest neighbours, standard neural networks, and Naïve Bayes are being conducted using realistic test data sets. The results demonstrate that VWCNNs have advantages over other classifiers in terms of classification accuracy and robustness. [ABSTRACT FROM AUTHOR]

• We propose an accurate heuristic to determine an upper limit to convolutional depth using Shannon's entropy measure for forced feature map compression and abstraction which reduces training time by 24.99%-78.59% across various CNN architectures and datasets. • Our experiments empirically validate and support the findings presented in [9] and [20] that, deep CNN models behave as a collection of ensemble networks and shallower CNN models can learn the same functional representations as deeper models with a reduction in the associated trade-off of relative model training time. Convolutional Neural Networks (CNNs) such as ResNet-50, DenseNet-40 and ResNeXt-56 are severely over-parameterized, necessitating a consequent increase in the computational resources required for model training which scales exponentially for increments in model depth. In this paper, we propose an Entropy-Based Convolutional Layer Estimation (EBCLE) heuristic which is robust and simple, yet effective in resolving the problem of over-parameterization with regards to network depth of CNN model. The EBCLE heuristic employs a priori knowledge of the entropic data distribution of input datasets to determine an upper bound for convolutional network depth, beyond which identity transformations are prevalent offering insignificant contributions for enhancing model performance. Restricting depth redundancies by forcing feature compression and abstraction restricts over-parameterization while decreasing training time by 24.99% - 78.59% without degradation in model performance. We present empirical evidence to emphasize the relative effectiveness of broader, yet shallower models trained using the EBCLE heuristic, which maintains or outperforms baseline classification accuracies of narrower yet deeper models. The EBCLE heuristic is architecturally agnostic and EBCLE based CNN models restrict depth redundancies resulting in enhanced utilization of the available computational resources. The proposed EBCLE heuristic is a compelling technique for researchers to analytically justify their HyperParameter (HP) choices for CNNs. Empirical validation of the EBCLE heuristic in training CNN models was established on five benchmarking datasets (ImageNet32, CIFAR-10/100, STL-10, MNIST) and four network architectures (DenseNet, ResNet, ResNeXt and EfficientNet B0-B2) with appropriate statistical tests employed to infer any conclusive claims presented in this paper. [ABSTRACT FROM AUTHOR]

• In this paper, we propose a variation of deep residual convolutional neural networks with robustness and efficiency in both learning and testing. • More importantly, we propose multiple strategies for model adaptation to the internal contents of the lowresolution input image and analyze their strong points and weaknesses. • Our adaptation especially favors images with repetitive structures or high resolutions. • We hope to arouse the interests of communities in focusing on internal priors, which are limited but have been proved effective and highly relevant. In recent years, impressive advances have been made in single-image super-resolution. Deep learning is behind much of this success. Deep(er) architecture design and external prior modeling are the key ingredients. The internal contents of the low-resolution input image are neglected with deep modeling, despite earlier works that show the power of using such internal priors. In this paper, we propose a variation of deep residual convolutional neural networks, which has been carefully designed for robustness and efficiency in both learning and testing. Moreover, we propose multiple strategies for model adaptation to the internal contents of the low-resolution input image and analyze their strong points and weaknesses. By trading runtime and using internal priors, we achieve improvements from 0.1 to 0.3 dB PSNR over the reported results on standard datasets. Our adaptation especially favors images with repetitive structures or high resolutions. It indicates a more practical usage when our adaption approach applies to sequences or videos in which adjacent frames are strongly correlated in their contents. Moreover, the approach can be combined with other simple techniques, such as back-projection and enhanced prediction, to realize further improvements. [ABSTRACT FROM AUTHOR]

• We propose a novel and lightweight Bi-RRNet for COD that includes an LRRN and a U-RRN. Both L-RRN and U-RRN iteratively refine the multilevel context features to obtain a highly discriminative high-resolution semantic feature representation for precise dense prediction. • We design two novel modules for intra- and inter-class feature enhancement including the MSPM and the RCEM. The MSPM uses global scene context signal to adjust the multi-scale features for each layer to mitigate the significant object appearance variations, while the RCEM is to highlight the camouflaged object region to enlarge the inter-class difference between the camouflaged objects and their surroundings. • Extensive evaluations on four challenging benchmark datasets demonstrate that our Bi-RRNet achieves superior performance against 15 state of-the-art methods in terms of both accuracy and model parameters. Besides, our Bi-RRNet is lightweight with only 14.95M model parameters that are only half of the State-Of-The-Art (SOTA) BSA-Net [26]. In this paper, we present a lightweight Bi-level Recurrent Refinement Network (Bi-RRNet) for Camouflaged Object Detection (COD) that consists of a Lower-level RRNet (L-RRN) and an Up-level RRNet (U-RRN) to progressively refine the multi-level context features for precise dense prediction. In particular, the L-RRN recursively refines the deeper layer high-level semantic features with the high-resolution low-level features from the earlier layers in a top-down manner, and the U-RRN progressively polishes the refined features from the L-RRN in a recurrent manner, producing the high-resolution semantic features that are essential to accurate COD. Moreover, we develop a Multi-scale Scene Perception Module (MSPM) that, in order to deal with target appearance variation, first compresses the global scene context information at each layer into a learnable weight vector and then modulates the multi-scale context features produced by a filter bank with various local receptive fields using the learned weights. Meanwhile, we design a Region-Consistency Enhancement Module (RCEM) that makes use of high-level semantic features to direct filtering out the cluttered information in the lower-layer features. This module can highlight the regions of camouflaged objects, maximizing the inter-class contrast between the objects and their surroundings. Extensive experiments on four challenging benchmark datasets, including CHAMELEON, CAMO, COD10K, and NC4K, show that our Bi-RRNet outperforms a variety of state-of-the-art methods in terms of accuracy and model parameters. Our Bi-RRNet, in particular, is lightweight, with 14.95M parameters that are only half the size of the state-of-the-art BSA-Net. [ABSTRACT FROM AUTHOR]

• The proposed composite network model seamlessly integrates the advantages of DCNNs and transformers to super-resolve LR face images. • The proposed Multi-Order Head Attention Network not only captures spatial and channel dependencies of facial priors, but it also models 2D information of face images. • The proposed model demonstrates competitive recovery performance in terms of visual results and quantitative evaluation, when compared with state-of-the-art FSR methods. Face super-resolution (FSR) aims to reconstruct high-resolution face images from low-resolution (LR) ones. Despite the progress made by deep convolutional neural networks (DCNNs) on FSR, convolutions struggle to relate spatially distant concepts and what is more, all image pixels and prior information (e.g., landmarks and facial component heatmaps) are treated equally regardless of importance, causing inaccuracy and decreasing the quality of face image recovery. To address these issues, in this paper we propose a composite network model for FSR with multi-order head attention facial priors. The proposed model contains a face hallucination transformer (FHT)-based network and a multi-order head attention (MOHA)-based DCNN. The FHT-based network can capture long-range dependencies and gradually increase resolution to achieve efficient and effective inference, while the MOHA-based DCNN exploits detailed and two-dimensional information of LR face images. Moreover, the novel generic submodule of the MOHA-based DCNN, namely Multi-Order Head Attention Network, can accurately model the relationship of facial components between spatial and channel dimensions. The proposed composite network model seamlessly integrates the advantages of DCNNs and transformers to super-resolve LR face images. When compared with state-of-the-art FSR methods on public benchmark datasets, the proposed model shows competitive recovery performance. [ABSTRACT FROM AUTHOR]

• We present "Deep Action Stamps (DeepActs)", a novel data representation which encodes actions in terms of spatial and motion information extracted from face, hands, and body. To the best of our knowledge, this is the first work which models the spatial and temporal dependencies between facial joints, hand joints, and body joints for action recognition. • We present DeepActsNet, an ensemble of Enhanced Convolutional Graph Networks (ECGN) that learn convolutional and structural features from different feature channels of Deep Actions Stamps. We also develop a lightweight strong baseline, which is more powerful than the previous methods in terms of recognition accuracy and computational efficiency. • We present ablation studies in terms of the benefits of combining spatial and motion information from face, hands, and body, and the significance of ensembling convolutional and structural features for improving accuracy of challenging action classes. Experiments on three public datasets show that our contributions consistently exceed the state-of-the-art performance on all datasets with considerable margins. Human action recognition from videos has gained substantial focus due to its wide applications in the field of video understanding. Most of the existing approaches extract human skeleton data from videos to encode actions because of the invariance nature of the skeleton information with respect to lightning conditions and background changes. Despite their success in achieving high recognition accuracy, methods based on limited body joints fail to capture the nuances of subtle body parts which are highly relevant for discriminating similar actions. In this paper, we overcome this limitation by presenting a holistic framework for combining spatial and motion features from the body, face, and hands to develop a novel data representation termed "Deep Actions Stamps (DeepActs)" for video-based action recognition. Compared to the skeleton sequences based on limited body joints, DeepActs encode more effective spatio-temporal features that provide robustness against pose estimation noises and improve action recognition accuracy. We also present "DeepActsNet", a deep learning based ensemble model which learns convolutional and structural features from Deep Action Stamps for highly accurate action recognition. Experiments on three challenging action recognition datasets (NTU60, NTU120, and SYSU) show that the proposed model produces significant improvements in the action recognition accuracy with less computational cost compared to the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

• The skewness and asymmetry in the relationship between channelwise responses and their importance. • An interpretable modeling on channelwise attention. • Consistent performance increase on various tasks and various backbones. • Visualization shows learned attention. Channelwise attention mechanisms have recently been demonstrated to boost the performance of deep convolutional neural networks (CNNs). The hypothesis on the negative correlation between channelwise responses and their importance levels has been verified. Therefore, according to the shifted means and unbalanced responses that are observed in attention distribution, two empirical hypotheses are proposed in this paper. Then, as an interpretable attention module, bilateral asymmetric skewed Gaussian attention (bi-SGA) is utilized to combine skewed and asymmetric properties into the Gaussian context transformer (GCT), which is the state-of-the-art. Finally, extensive experiments on the different benchmarks validate the rationality of the hypotheses. The proposed bi-SGA improves the overall performance of GCT attention mechanisms with only two extra parameters to be learned. Moreover, our attention mechanism also provides a perspective on analyzing the channelwise importance levels in deep neural networks in an interpretable and logical manner. [ABSTRACT FROM AUTHOR]

• We propose a novel deep neural network based on Transformer, named TransHRNet, which connects the different resolution streams in parallel and repeatedly exchanges the information across resolutions. • An Effective Transformer (EffTrans) is introduced to promote the performance, which uses the group linear transformations with an expand-reduce strategy and the spatial-reduction attention layer to further reduce the resource cost. • Our proposed method achieves higher performance than SoTA efficient medical image segmentation method with comparable computation cost. Most recent 3D medical image segmentation methods adopt convolutional neural networks (CNNs) that rely on deep feature representation and achieve adequate performance. However, due to the convolutional architectures having limited receptive fields, they cannot explicitly model the long-range dependencies in the medical image. Recently, Transformer can benefit from global dependencies using self-attention mechanisms and learn highly expressive representations. Some works were designed based on the Transformers, but the existing Transformers suffer from extreme computational and memories, and they cannot take full advantage of the powerful feature representations in 3D medical image segmentation. In this paper, we aim to connect the different resolution streams in parallel and propose a novel network, named Trans former based H igh R esolution Net work (TransHRNet), with an Effective Transformer (EffTrans) block, which has sufficient feature representation even at high feature resolutions. Given a 3D image, the encoder first utilizes CNN to extract the feature representations to capture the local information, and then the different feature maps are reshaped elaborately for tokens that are fed into each Transformer stream in parallel to learn the global information and repeatedly exchange the information across streams. Unfortunately, the proposed framework based on the standard Transformer needs a huge amount of computation, thus we introduce a deep and effective Transformer to deliver better performance with fewer parameters. The proposed TransHRNet is evaluated on the Multi-Atlas Labeling Beyond the Cranial Vault (BCV) dataset that consists of 11 major human organs and the Medical Segmentation Decathlon (MSD) dataset for brain tumor and spleen segmentation tasks. Experimental results show that it performs better than the convolutional and other related Transformer-based methods on the 3D multi-organ segmentation tasks. Code is available at https://github.com/duweidai/TransHRNet. [ABSTRACT FROM AUTHOR]

• A novel Transformer-based architecture for very low resolution images object detection is proposed. • An image down-sampling module that utilizes the feature extraction capabilities of CNNs to generate thumbnail images from the original ones. • A distillation-boost supervision strategy is introduced to maintain the detection performance of thumbnail images as the original-size inputs. • Our method achieves a satisfactory detection performance (i.e. 32.3% in mAP) while drastically reduce computation and memory requirements (speed up 1.26 ×). • The performance of our method outperforms bicubic method by a large margin (3.2% in mAP) and achieves comparable results compared to SOTA method on COCO dataset. Computer vision fields have witnessed great success thanks to deep convolutional neural networks (CNNs). However, state-of-the-art methods often benefit from large models and datasets, which introduce heavy parameters and computational requirements. Deploying such large models in real-world applications is very difficult because of the limited computing resources. Although many researchers focus on designing efficient block structures to compress model parameters, they ignore that the role of large-scale input images is also an important factor for algorithm efficiency. Reducing input resolution is a useful method to boost runtime efficiency, however, traditional interpolation methods assume a fixed degradation criterion that greatly hurts performance. To solve the above problems, in this paper, we propose a novel framework named ThumbDet for reducing model computation while maintaining detection accuracy. In our framework, we first design an image down-sampling module to learn a small-scale image that looks realistic and contains discriminative properties. Furthermore, we propose a distillation-boost supervision strategy to maintain the detection performance of small-scaled images as the original-size inputs. Extensive experiments conducted on a standard object detection dataset MS COCO demonstrate the effectiveness of the proposed method when using very low-resolution images (i.e. 4 × down-sampling) as inputs. In particular, ThumbDet achieves satisfactory detection performance (i.e. 32.3% in mAP) while drastically reducing computation and memory requirements (i.e. speed up of 1.26 ×), outperforming the traditional interpolation methods (e.g. bicubic) by +3.2% absolutely in terms of mAP. [ABSTRACT FROM AUTHOR]

• A novel deep CNN architecture is proposed for jointly learning the expert consensus label and the annotator's label. The proposed architecture (Fig. 1) consists of two coupled CNNs where one estimates the expert consensus label probabilities and the other models the characteristics of individual annotators (e.g., tendency to over-segmentation, mix-up between different classes, etc) by estimating the pixel-wise confusion matrices (CMs) on a per image basis. Unlike STAPLE [25] and its variants, our method models, and disentangles with deep neural networks, the complex mappings from the input images to the annotator behaviours and to the expert consensus label. • The parameters of our CNNs are "global variables" that are optimised across different image samples; this enables the model to disentangle robustly the annotators' mistakes and the expert consensus label based on correlations between similar image samples, even when the number of available annotations is small per image (e.g., a single annotation per image). In contrast, this would not be possible with STAPLE [25] and its variants [5,8] where the annotators' parameters are estimated on every target image separately. • This paper extends the preliminary version of our method presented at the NeurIPS Thirty-fourth Annual Conference on Neural Information Processing Systems [30] , by extensively evaluating our model on a new created real-world multiple sclerosis lesion dataset (QSMSC at UCL: Queen Square Multiple Sclerosis Center at UCL, UK). This dataset is generated with manual segmentations from 4 different annotators (3 radiologists with different level skills and 1 expert to generate the expert consensus label). Additionally, we presented a comprehensive discussion about our model's potential applications (e.g., estimate annotator's quality and annotation's quality), the future works we are going to explore, and the potential limitations of our model. [Display omitted] Supervised machine learning methods have been widely developed for segmentation tasks in recent years. However, the quality of labels has high impact on the predictive performance of these algorithms. This issue is particularly acute in the medical image domain, where both the cost of annotation and the inter-observer variability are high. Different human experts contribute estimates of the "actual" segmentation labels in a typical label acquisition process, influenced by their personal biases and competency levels. The performance of automatic segmentation algorithms is limited when these noisy labels are used as the expert consensus label. In this work, we use two coupled CNNs to jointly learn, from purely noisy observations alone, the reliability of individual annotators and the expert consensus label distributions. The separation of the two is achieved by maximally describing the annotator's "unreliable behavior" (we call it "maximally unreliable") while achieving high fidelity with the noisy training data. We first create a toy segmentation dataset using MNIST and investigate the properties of the proposed algorithm. We then use three public medical imaging segmentation datasets to demonstrate our method's efficacy, including both simulated (where necessary) and real-world annotations: 1) ISBI2015 (multiple-sclerosis lesions); 2) BraTS (brain tumors); 3) LIDC-IDRI (lung abnormalities). Finally, we create a real-world multiple sclerosis lesion dataset (QSMSC at UCL: Queen Square Multiple Sclerosis Center at UCL, UK) with manual segmentations from 4 different annotators (3 radiologists with different level skills and 1 expert to generate the expert consensus label). In all datasets, our method consistently outperforms competing methods and relevant baselines, especially when the number of annotations is small and the amount of disagreement is large. The studies also reveal that the system is capable of capturing the complicated spatial characteristics of annotators' mistakes. [ABSTRACT FROM AUTHOR]

• First in-depth experimental study of Transformer Networks and BERT for trajectory forecasting. • Comparative evaluation on the modelling of individual human trajectories. • First exhaustive evaluation of input/output representations and problem formulations. • Focus on the capability of Transformers to predict multi-modal futures. • Analysis of the impact of intention and long-term forecasting. Transformer Networks have established themselves as the de-facto state-of-the-art for trajectory forecasting but there is currently no systematic study on their capability to model the motion patterns of people, without interactions with other individuals nor the social context. There is abundant literature on LSTMs, CNNs and GANs on this subject. However methods adopting Transformer techniques achieve great performances by complex models and a clear analysis of their adoption as plain sequence models is missing. This paper proposes the first in-depth study of Transformer Networks (TF) and the Bidirectional Transformers (BERT) for the forecasting of the individual motion of people, without bells and whistles. We conduct an exhaustive evaluation of the input/output representations, problem formulations and sequence modelling, including a novel analysis of their capability to predict multi-modal futures. Out of comparative evaluation on the ETH+UCY benchmark, both TF and BERT are top performers in predicting individual motions and remain within a narrow margin wrt more complex techniques, including both social interactions and scene contexts. Source code will be released for all conducted experiments. [ABSTRACT FROM AUTHOR]

• LASA fills the gap that no adversary is designed specified for sonar image, according to the real-world underwater conditions. • The performance of LASA is superior to the existing state-of-the-art adversarial attack methods (both targeted and non-targeted) in SSS image classification. • We demonstrate that the adversarial training indeed improves the robustness of CNNs against several adversarial attack methods. • LASA shows better understanding of adversarial attack on SSS images. Deep convolutional neural networks (CNNs) are extensively applied to the classification tasks for Side-scan sonar (SSS) images. However, state-of-the-art neural networks are prone to be confused by adversarial attacks that generate a tiny modification of the images, threatening the security of SSS classification. The robustness of CNN to adversarial attacks can be improved by introducing adversarial examples through adversarial training. Practical adversarial examples are often generated from elaborate adversarial attackers. For the underwater scenario of sonar, a specially designed adversarial attack method to weaken SSS image classification can make the research community better understand the weakness of CNN in this scenario and improve the security measures in a well-directed way. Thus, exploring adversarial attack methods for SSS image classification is essential. Nevertheless, the existing adversarial attack methods are designed for optical images, reflecting no physical characteristics of sonar images. To fill this gap and investigate the adversarial attack related to real-world conditions, in this paper, we propose an adversarial attack method named Lambertian Adversarial Sonar Attack (LASA). It initially leverages the Lambertian model to simulate the formation of the SSS image, factoring the image to three parameters, then updates the parameters on the direction of gradients by the chain rule. Finally, the parameters regenerate the adversarial example to fool the classifier. To validate the performance of LASA, we constructed a diversified SSS image dataset containing three categories. On our dataset, LASA reduces the Top-1 accuracy of a well-trained ResNet-101 to 7.31 % ± 0.21 (one-shot version) and 0.00% (iterative version), the success rate of targeted attack reaches 97.03 ± 2.24 , far beyond the performance of the existing state-of-the-art adversarial attack methods. Meanwhile, we show that the adversarial training using examples generated from LASA makes the classifier more robust. We expect that our methods can be applied as a benchmark of adversarial attacks on SSS images, motivating future research to design novel neural networks or defensive methods to resist real-world adversarial attacks on SSS images. [ABSTRACT FROM AUTHOR]

• An effective depth structure completion method is developed to jointly infer missing contents in flat regions and depth structures. • The texture-copy artifacts induced by RGB textures are well suppressed by a kernel least-square module. • The proposed method considerably outperforms recent baselines and exhibits good generalization ability. Raw depth maps captured by depth sensors generally contain missing contents due to glossy, transparent, and sparsity problems. Recent methods well completed flat regions of raw depth maps; however, ignored the accuracy of depth structures. In this paper, an effective depth structure completion method is developed to infer missing depth structures. First, a raw depth map is divided into flat regions and depth structures based on a structure prediction network. Second, two local features including surface normals and Gaussian weights are extracted from a reference RGB image to impose constraints on flat regions and depth structures, separately. Third, a kernel least-square module is adopted to handle the texture-copy artifacts problem. Finally, an iterative optimization model is developed by embedding the two constraints into a Markov random field. The cost function of the model comprises three terms, which limit data fidelity between completed depth map and raw depth map, smoothness of flat regions, and accuracy of depth structures, respectively. The proposed method is evaluated on four indoor datasets including Matterport3D, RealSense, ScanNet, and NYUv2, and compared with eight recent baselines. Quantitative results demonstrate that RMSE and MAE of completed depth maps are considerably reduced by 22.0% and 45.3%, respectively. Visual results show the superiority in completing depth structures and suppressing texture-copy artifacts. Generalization test verify the effectiveness on unseen datasets. [ABSTRACT FROM AUTHOR]

• Present an edge detection method with refined network by solving low-quality detection (non-maximum pixels and the noise around the edge region). • Batch-normalization is used with learnable affine parameters which make the proposed network able to remove these kinds of noise around the edge region. • Network used multi-scale representation with high resolution detection and avoiding the loss of certain features during pooling operations. Edge detection is represented as one of the most challenging tasks in computer vision, due to the complexity of detecting the edges or boundaries in real-world images that contains objects of different types and scales like trees, building as well as various backgrounds. Edge detection is represented also as a key task for many computer vision applications. Using a set of backbones as well as attention modules, deep-learning-based methods improved the detection of edges compared with traditional methods like Sobel or Canny. However, images of complex scenes still represent a challenge for these methods. Also, the detected edges using the existing approaches suffer from non-refined results with erroneous edges. In this paper, we attempted to overcome these challenges for refined edge detection using a cascaded and high-resolution network named (CHRNet). By maintaining the high resolution of edges during the training process, and conserving the resolution of the edge image during the network stage, sub-blocks are connected at every stage with the output of the previous layer. Also, after each layer, we use batch normalization layer with an active affine parameter as an erosion operation for the homogeneous region in the image. The proposed method is evaluated using the most challenging datasets including BSDS500, NYUD, and Multicue. The obtained results outperform the designed edge detection networks in terms of performance metrics and quality of output images.The code is available at: https://github.com/elharroussomar/chrnet/ [ABSTRACT FROM AUTHOR]

• We propose a novel perspective of how to decrease feature redundancy for UDA based on the feature spectrum to align the two domains. As far as we know, this is the first work to look into feature redundancy for UDA. • A cross-domain batch normalization with whitening module (xBN) is devised to reduce domain-specific feature redundancy for better generalization, which can be easily be incorporated to any BN-based backbone networks for UDA. • A novel orthogonal regularization is proposed to equalize domain-invariant feature spectrum, which can realize the orthogonality to further decrease the feature redundancy. Unsupervised domain adaptation (UDA) deals with the problem of transferring knowledge from a labeled source domain to an unlabeled target domain when the two domains have distinct data distributions. Therefore, the purpose of domain adaptation is to mitigate the distribution divergence between the two domains. Many existing UDA methods only use the traditional batch normalization layer, but this may lead to a large number of feature redundancy and lead performance degradation. In this paper, we introduce a novel deep learning paradigm called feature redundancy in UDA to enhance adaptation ability. Specifically, we first show that feature redundancy also exists on unsupervised domain adaptation (UDA), which has been ignored by most previous efforts. We utilize feature similarity as a metric to measure feature redundancy and then analyze the relationship between uniform feature spectrum and minimal feature similarity. Based on this relationship, we intend to reduce cross-domain feature redundancy for UDA by making the distribution of feature spectrum uniforms in a bi-level way. For the first level, we propose a cross-domain batch normalization with the whitening module (xBN) to ensure compact domain-specific features and learn domain-invariant features at the same time. With the domain-specific features from the first level that paves a way, on the second level, we suggest an alternative orthogonal regularizer (OR) that can make the distribution of the feature spectrum more uniform, thus domain-invariant feature redundancy is mitigated. Such a bi-level mechanism greatly reduces the feature redundancy for UDA. To evaluate the efficacy of the proposed bi-level mechanism, we plug those two novel modules (i.e. , xBN and OR) into convolutional neural networks (CNNs) to form our UDA model and also conduct the corresponding empirical evaluations on five cross-domain object recognition benchmarks including both classical and large-scale image datasets. Experimental results show that the proposed UDA model could achieve state-of-the-art performance both in quantity and quality. Our source codes will be released after publication. [ABSTRACT FROM AUTHOR]