Search

Although current methods based on deep learning (DL) have been widely employed in shadow detection tasks, the cluttered background and complex shadow features in remote sensing images (RSIs) make shadow detection still a challenging task. In this paper, we apply a neural network combined with a distance transformation algorithm to RSI shadow detection for the first time and propose a novel slice-to-slice context transfer and uncertainty region calibration network (SCUCNet). First, an interslice context semantic transfer (SCT) module is proposed to explore the connections among spatial pixel information and extract sufficient global contextual information via interslice feature transfer. This process helps alleviate the interference of multiscale differences and complex backgrounds on the detection results. Second, to further optimize the initial output mapping, calibrate the uncertainty region, and enhance the feature representation, we propose a plug-and-play uncertainty region awareness and calibration (URAC) module combined with a distance transform algorithm, which introduces only 0.09 M parameters. The quantitative results show that the proposed model outperforms state-of-the-art methods for shadow detection in RSIs, achieving an Intersection over Union (IoU) of 88.33% on the aerial imagery dataset for shadow detection (AISD). In addition, extensive experiments and visualizations further prove the validity and interpretability of our method, and the proposed method can effectively mitigate the interference caused by multiscale differences, morphological diversity, spectral similarity and heterogeneity. [ABSTRACT FROM AUTHOR]

• Intention inference problem is formulated as a time-series learning problem. • A deep learning-based model is proposed for intention inference. • The intention space set is established based on typical space relative motions. • Self-attention mechanism captures correlation in time-series to improve efficiency. • Our model has better performance and robustness than existing models. Intention inference for space non-cooperative targets is the key to space situational awareness and assistant decision for collision avoidance. Given that the problem of target intention inference is essential to learn the dynamically changing time-series characteristics of space non-cooperative target intentions and infer their relative motion patterns for threat warning, this paper adopts a deep learning-based approach, introduces a bidirectional propagation mechanism and self-attention mechanism based on Gated Recurrent Unit (GRU) and proposes a bidirectional Gated Recurrent Unit (BiGRU)-Self Attention-based space non-cooperative target intention inference model. BiGRU is used to learn deep information in time-series characteristics of the space non-cooperative target, and self-attention mechanism is used to adaptively extract and assign weights to key characteristics to capture the internal correlations in time-series information, thus improving model performance. The line-of-sight measurements are used as the characteristics of target intention inference, and the typical target motion intentions are defined. Subsequently, the proposed model is trained and tested on the test set, with the accuracy reaching 97.1%. Besides, the effectiveness and advantages of the proposed model are verified by the simulation of a case study and comparison evaluations. The results demonstrate that our proposed model could significantly improve the accuracy, computational efficiency, and noise resistance for the space non-cooperative target intention inference compared with the existing intention inference models. [ABSTRACT FROM AUTHOR]

During the last few years, many deep learning (DL) methods have been proposed for the classification of hyperspectral images (HSI), and the final results show that these models have great potential. However, most existing deep models only consider the original spectral HSI datacube, excluding other sources of information such as textural and geometrical features. In addition to their computational complexity, most DL approaches cannot effectively consider local nonlinear relationships between features. In order to address these issues, this paper develops a new DL approach called MLKD-CNN, which uses multifeature local kernel descriptors (MLKDs) and convolutional neural networks (CNN) to classify HSIs. This method produces local kernel descriptors (LKDs) of spectral, extended morphological profiles, and Gabor features in the feature generation step. A simple CNN is then used to classify MLKD generated from spatial and spectral features. The proposed MLKD considers local nonlinear relationships of both spectral and spectral features, making them capable of achieving high accuracy levels while reducing the computational burden of deep methods. The experiments on four benchmark HSIs show that the proposed MLKD-CNN outperforms some recent HSI classification methods in terms of accuracy and computation time. [ABSTRACT FROM AUTHOR]

Missions to small celestial bodies rely heavily on optical feature tracking for characterization of and relative navigation around the target body. While deep learning has led to great advancements in feature detection and description, training and validating data-driven models for space applications is challenging due to the limited availability of large-scale, annotated datasets. This paper introduces AstroVision, a large-scale dataset comprised of 115,970 densely annotated, real images of 16 different small bodies captured during past and ongoing missions. We leverage AstroVision to develop a set of standardized benchmarks and conduct an exhaustive evaluation of both handcrafted and data-driven feature detection and description methods. Next, we employ AstroVision for end-to-end training of a state-of-the-art, deep feature detection and description network and demonstrate improved performance on multiple benchmarks. The full benchmarking pipeline and the dataset will be made publicly available to facilitate the advancement of computer vision algorithms for space applications. • Missions to small bodies rely heavily on humans for optical feature tracking. • Deep Learning has led to great advancements in feature detection and description. • We present AstroVision, a first-of-a-kind dataset for vision-based tasks near small bodies. • We conduct an extensive evaluation of feature detection and description methods. • We demonstrate increased performance by training a deep feature network on our data. [ABSTRACT FROM AUTHOR]

Neural architecture search (NAS) is a framework for automating the design process of a neural network structure. While the recent one-shot approaches have reduced the search cost, there still exists an inherent trade-off between cost and performance. It is important to appropriately stop the search and further reduce the high cost of NAS. Meanwhile, the differentiable architecture search (DARTS), a typical one-shot approach, is known to suffer from overfitting. Heuristic early-stopping strategies have been proposed to overcome such performance degradation. In this paper, we propose a more versatile and principled early-stopping criterion on the basis of the evaluation of a gap between expectation values of generalisation errors of the previous and current search steps with respect to the architecture parameters. The stopping threshold is automatically determined at each search epoch without cost. In numerical experiments, we demonstrate the effectiveness of the proposed method. We stop the one-shot NAS algorithms and evaluate the acquired architectures on the benchmark datasets: NAS-Bench-201 and NATS-Bench. Our algorithm is shown to reduce the cost of the search process while maintaining a high performance. [ABSTRACT FROM AUTHOR]

With weight-sharing and continuous relaxation strategies, the differentiable architecture search (DARTS) proposes a fast and effective solution to perform neural network architecture search in various deep learning tasks. However, unresolved issues, such as the inefficient memory utilization, and the poor stability of the search architecture due to channels randomly selected, which has even caused performance collapses, are still perplexing researchers and practitioners. In this paper, a novel efficient channel attention mechanism based on partial channel connection for differentiable neural architecture search, termed EPC-DARTS, is proposed to address these two issues. Specifically, we design an efficient channel attention module, which is applied to capture cross-channel interactions and assign weight based on channel importance, to dramatically improve search efficiency and reduce memory occupation. Moreover, only partial channels with higher weights in the mixed calculation of operation are used through the efficient channel attention mechanism, and thus unstable network architectures obtained by the random selection operation can also be avoided in the proposed EPC-DARTS. Experimental results show that the proposed EPC-DARTS achieves remarkably competitive performance (CIFAR-10/CIFAR-100: a test accuracy rate of 97.60%/84.02%), compared to other state-of-the-art NAS methods using only 0.2 GPU-Days. [ABSTRACT FROM AUTHOR]

Despite the enormous achievements of Deep Learning (DL) based models, their non-transparent nature led to restricted applicability and distrusted predictions. Such predictions emerge from erroneous In-Distribution (ID) and Out-Of-Distribution (OOD) samples, which results in disastrous effects in the medical domain, specifically in Medical Image Segmentation (MIS). To mitigate such effects, several existing works accomplish OOD sample detection; however, the trustworthiness issues from ID samples still require thorough investigation. To this end, a novel method TrustMIS (Trustworthy Medical Image Segmentation) is proposed in this paper, which provides the trustworthiness and improved performance of ID samples for DL-based MIS models. TrustMIS works in three folds: IT (Investigating Trustworthiness), INT (Improving Non-Trustworthy prediction) and CSO (Classifier Switching Operation). Initially, the IT method investigates the trustworthiness of MIS by leveraging similar characteristics and consistency analysis of input and its variants. Subsequently, the INT method employs the IT method to improve the performance of the MIS model. It leverages the observation that an input providing erroneous segmentation can provide correct segmentation with rotated input. Eventually, the CSO method employs the INT method to scrutinise several MIS models and selects the model that delivers the most trustworthy prediction. The experiments conducted on publicly available datasets using well-known MIS models reveal that TrustMIS has successfully provided a trustworthiness measure, outperformed the existing methods, and improved the performance of state-of-the-art MIS models. Our implementation is available at https://github.com/SnehaShukla937/TrustMIS. [ABSTRACT FROM AUTHOR]

The goal of face attribute recognition(FAR) is to recognize the attributes of face images, such as gender, race, etc. Multi-dataset fusion aims to train a network with multiple datasets simultaneously, which has significant advantages in deep learning. For example, in FAR, the attribute labels differ with the datasets. When using multi-dataset fusion on FAR, training data and attribute labels can be increased so that the network performance can be improved and more attributes can be recognized simultaneously. Currently, most related researchers take the multi-task learning approach on FAR, where each attribute is treated as a recognition task. However, the existing multi-task learning framework cannot be applied when using multi-dataset fusion on FAR. That is because multi-task learning requires the training data to contain the labels of all tasks, but no data contains all the attribute labels of multiple datasets. A Multi-Dataset and Multi-Task Framework(MDMTF) that takes knowledge distillation as the core is proposed in this paper to deal with the problem above. Three distillation strategies are designed in the MDMTF according to the characteristics of multi-dataset fusion on FAR. Experimental results on CelebA and MAAD-Face datasets demonstrate the effectiveness of our framework and strategies. Compared to training two networks on both datasets, respectively, that is, single dataset training, our method has significant advantages in accuracy, parameters, and computational complexity. • Multi-dataset fusion is valuable for face attribute recognition. • The existing multi-task learning framework cannot be applied to this case. • A framework with knowledge distillation is proposed for this case. • Our framework is applicable to multi-dataset fusion on other multi-task fields. [ABSTRACT FROM AUTHOR]

Text-to-image (T2I) generation, which involves synthesizing an image from a textual description, has emerged as a popular research topic in computer vision. Meanwhile, transformer-based models, such as BERT, GPT-2, and T5, have demonstrated promising results in various natural language processing tasks, including text generation and translation. However, their application to T2I generation remains largely unexplored. Therefore, a comparative study investigating the performance of BERT, GPT-2, and T5 in T2I generation is of significant importance. Such a study would shed light on the strengths and weaknesses of each model, facilitating the identification of the most suitable approach for this task. In this paper, we propose three architectures to conduct a comparative study of T5, GPT-2, and BERT in T2I generation tasks. We fine-tune these models to generate text vectors and then transform the textual information into images using the affine transformation into the DF-GAN generator. Subsequently, we evaluate the quality, diversity, and the models' ability to recognize the words. Our experiments on the challenging CUB and Oxford-102 flower datasets demonstrate that T5 exhibits promising potential for T2I generation. It has the capability to generate visually appealing and semantically coherent images from textual descriptions. • GPT-2, BERT, and T5 have succeeded in synthesizing the images from the textual description with varying degrees of success. • BERT allows the model to capture the relationships between words in the input text and generate a more coherent image. • GPT-2 allows the model to generate plausible images, but the quality of the images is often lower compared to other models. • T5 allows the model to be more flexible, it is the powerful transformer-based model for text-to-image generation tasks. • The quality of the generated images is still lower compared to other generation models. [ABSTRACT FROM AUTHOR]

Pipeline leakage and water deposits can cause serious consequences, such as environmental pollution, safety accidents, and economic losses. Therefore, effective detection of these flaws is of critical importance. Currently, most of the detection methods rely heavily on experienced inspectors and specialized equipment, which is labor-intensive and costly. To this end, this paper presents a one-dimensional residual convolutional neural network (1D-ResNet) based percussion method, for detecting pipeline leakage and water deposit. The proposed method uses sound produced by tapping the pipe as input to 1D-ResNet, which can directly extract features from the audio signal, avoiding hand-crafted feature extraction process. The effectiveness of the proposed method is validated through experiments, showing strong performance in pipeline fault detection. Furthermore, the 1D-ResNet method exhibits better classification performance and stronger noise robustness compared to other methods. In summary, this study presents a novel approach for the detection of pipeline leakage and deposit through the innovative introduction of 1D-ResNet deep learning technology, which has significant application prospects. [ABSTRACT FROM AUTHOR]

Water management failure is the most common fault in proton-exchange membrane fuel cells (PEMFCs), and it directly affects the durability and stability of fuel cells. This paper proposes a water fault diagnosis method based on 1DCNN-XGB. To promote its commercial applications, the cathode pressure drop, voltage, and current density were used as the characteristic diagnostic variables，which considered variations in the stack load and also facilitated hierarchical fault diagnosis. First, flooding and drying experiments under different current densities were simulated by changing the operating conditions of the stack, and the obtained experimental data were normalized to eliminate feature imbalances. Then, they were reconstructed into a 1D data set as the input of the model. Automatic feature extraction was performed by a 1DCNN, and the extracted feature maps were used as the input of the XGBoost classifier. Finally, the trained model was validated on the test set for fault diagnosis. The experimental results showed that the model accurately and efficiently distinguished normal and different degrees of two typical fault states (flooding and drying) of the stack with an overall accuracy of 98.10%. The comparative experiments revealed that the model was superior to the individual 1DCNN and XGBoost models, showing greater accuracy and better generalization ability. • 1DCNN–XGBoost was first used for fault diagnosis in PEMFC. • Fault hierarchy diagnosis under variation in load was studied, which is closer to commercial applications. • The cathode pressure drop and voltage reflected the flooding, drying law, and the degree of severity. • The proposed model can effectively detect flooding, drying and different degrees of faults under load variation. [ABSTRACT FROM AUTHOR]

Scene understanding has made tremendous progress over the past few years, as data acquisition systems are now providing an increasing amount of data of various modalities (point cloud, depth, RGB...). However, this improvement comes at a large cost on computation resources and data annotation requirements. To analyze geometric information and images jointly, many approaches rely on both a 2D loss and 3D loss, requiring not only 2D per pixel-labels but also 3D per-point labels. However, obtaining a 3D groundtruth is challenging, time-consuming and error-prone. In this paper, we show that image segmentation can benefit from 3D geometric information without requiring a 3D groundtruth, by training the geometric feature extraction and the 2D segmentation network jointly, in an end-to-end fashion, using only the 2D segmentation loss. Our method starts by extracting a map of 3D features directly from a provided point cloud by using a lightweight 3D neural network. The 3D feature map, merged with the RGB image, is then used as an input to a classical image segmentation network. Our method can be applied to many 2D segmentation networks, improving significantly their performance with only a marginal network weight increase and light input dataset requirements, since no 3D groundtruth is required. • Geometry. • Computer vision. • 2D segmentation. • Neural network. • multi-modalities. [ABSTRACT FROM AUTHOR]

Camera-based indoor localization is a fundamental aspect of indoor navigation, virtual reality, and location-based services. Deep learning methods have exhibited remarkable performance with low storage requirements and high efficiency. However, existing methods mainly derive features implicitly for pose regression without considering explicit structure information from images. This paper proposes that incorporating such information can improve the localization performance of learning-based approaches. We extract structure information from RGB images in the form of depth maps and edge maps and design two modules for depth-weighted and edge-weighted feature fusion. These modules are integrated into the pose regression network to enhance pose prediction. Furthermore, we employ a self-attention module for high-level feature extraction to augment the network capacity. Extensive experiments are conducted on the publicly available 7Scenes and 12Scenes datasets, and the results demonstrate that the proposed method achieves high localization performance, with an average positional error of 0. 19 m and 0. 16 m , respectively. The code for this work is available at https://github.com/lqing900205/structureLoc. [ABSTRACT FROM AUTHOR]

Over the past few years, single image super-resolution (SR) has become a hotspot in the remote sensing area, and numerous methods have made remarkable progress in this fundamental task. However, they usually rely on the assumption that images suffer from a fixed known degradation process, e.g., bicubic downsampling. To save us from performance drop when real-world distribution deviates from the naive assumption, blind image super-resolution for multiple and unknown degradations has been explored. Nevertheless, the lack of a real-world dataset and the challenge of reasonable degradation estimation hinder us from moving forward. In this paper, a self-supervised degradation-guided adaptive network is proposed to mitigate the domain gap between simulation and reality. Firstly, the complicated degradations are characterized by robust representations in embedding space, which promote adaptability to the downstream SR network with degradation priors. Specifically, we incorporated contrastive learning to blind remote sensing image SR, which guides the reconstruction process by encouraging the positive representations (relevant information) while punishing the negatives. Besides, an effective dual-wise feature modulation network is proposed for feature adaptation. With the guide of degradation representations, we conduct modulation on feature and channel dimensions to transform the low-resolution features into the desired domain that is suitable for reconstructing high-resolution images. Extensive experiments on three mainstream datasets have demonstrated our superiority against state-of-the-art methods. Our source code can be found at https://github.com/XY-boy/DRSR • A simple and effective self-supervised degradation representation learning strategy is proposed. • The learned representations can promote cross-domain super-resolution generalization. • A dual-wise modulation network is established for effective feature adaption. • Our method achieves favorable performance on various remote sensing datasets. [ABSTRACT FROM AUTHOR]

Over the years, Machine Learning models have been successfully employed on neuroimaging data for accurately predicting brain age. Deviations from the healthy brain aging pattern are associated with the accelerated brain aging and brain abnormalities. Hence, efficient and accurate diagnosis techniques are required to elicit accurate brain age estimations. Several contributions have been reported in the past for this purpose, resorting to different data-driven modeling methods. Recently, deep neural networks (also referred to as deep learning) have become prevalent in manifold neuroimaging studies, including brain age estimation. In this review, we offer a comprehensive analysis of the literature related to the adoption of deep learning for brain age estimation with neuroimaging data. We detail and analyze different deep learning architectures used for this application, pausing at research works published to date quantitatively exploring their application. We also examine different brain age estimation frameworks, comparatively exposing their advantages and weaknesses. Finally, the review concludes with an outlook towards future directions that should be followed by prospective studies. The ultimate goal of this paper is to establish a common and informed reference for newcomers and experienced researchers willing to approach brain age estimation by using deep learning models. • We categorize brain age estimation models based on the deep learning architectures. • We classify deep learning architectures based on the slice/voxel-based input data. • We examine the modalities, size of data, performance and the datasets used. • We enumerate several challenges and trace directions that can be explored further. [ABSTRACT FROM AUTHOR]

Graph Convolutional Networks (GCNs) with naive message passing mechanisms have limited performance due to the isotropic aggregation strategy. To remedy this drawback, some recent works focus on how to design anisotropic aggregation strategies with tricks on feature mapping or structure mining. However, these models still suffer from the low ability of expressiveness and long-range modeling for the needs of high performance in practice. To this end, this paper proposes a tree-guided anisotropic GCN, which applies an anisotropic aggregation strategy with competitive expressiveness and a large receptive field. Specifically, the anisotropic aggregation is decoupled into two stages. The first stage is to establish the path of the message passing on a tree-like hypergraph consisting of substructures. The second one is to aggregate the messages with constrained intensities by employing an effective gating mechanism. In addition, a novel anisotropic readout mechanism is constructed to generate representative and discriminative graph-level features for downstream tasks. Our model outperforms baseline methods and recent works on several synthetic benchmarks and datasets from different real-world tasks. In addition, extensive ablation studies and theoretical analyses indicate the effectiveness of our proposed method. [ABSTRACT FROM AUTHOR]

To learn the embedding representation of graph structure data corrupted by noise and outliers, existing graph structure learning networks usually follow the two-step paradigm, i.e., constructing a "good" graph structure and achieving the message passing for signals supported on the learned graph. However, the data corrupted by noise may make the learned graph structure unreliable. In this paper, we propose an adaptive graph convolutional clustering network that alternatively adjusts the graph structure and node representation layer-by-layer with back-propagation. Specifically, we design a Graph Structure Learning layer before each Graph Convolutional layer to learn the sparse graph structure from the node representations, where the graph structure is implicitly determined by the solution to the optimal self-expression problem. This is one of the first works that uses an optimization process as a Graph Network layer, which is obviously different from the function operation in traditional deep learning layers. An efficient iterative optimization algorithm is given to solve the optimal self-expression problem in the Graph Structure Learning layer. Experimental results show that the proposed method can effectively defend the negative effects of inaccurate graph structures. The code is available at https://github.com/HeXiax/SSGNN. [ABSTRACT FROM AUTHOR]

The past decade has witnessed significant progress in detecting objects by using enormous features of deep learning models. But, most of the existing models are unable to detect x-small and dense objects, due to the futility of feature extraction, and substantial misalignments between anchor boxes and axis-aligned convolution features, which leads to the discrepancy between the categorization score and positioning accuracy. This paper introduces an anchor regenerative-based transformer module in a feature refinement network to solve this problem. The anchor-regenerative module can generate anchor scales based on the semantic statistics of the objects present in the image, which avoids the inconsistency between the anchor boxes and axis-aligned convolution features. Whereas, the Multi-Head-Self-Attention (MHSA) based transformer module extracts the in-depth information from the feature maps based on the query, key, and value parameter information. This proposed model is experimentally verified on the VisDrone, VOC, and SKU-110K datasets. This model generates different anchor scales for these three datasets and achieves higher mAP, precision, and recall values on three datasets. These tested results prove that the suggested model has outstanding achievements compared with existing models in detecting x-small objects as well as dense objects. Finally, we evaluated the performance of these three datasets by using accuracy, kappa coefficient, and ROC metrics. These evaluated metrics demonstrate that our model is a good fit for VOC, and SKU-110K datasets. [ABSTRACT FROM AUTHOR]

The backpropagation algorithm has promoted the rapid development of deep learning, but it relies on a large amount of labeled data and still has a large gap with how humans learn. The human brain can quickly learn various conceptual knowledge in a self-organized and unsupervised manner, accomplished through coordinating various learning rules and structures in the human brain. Spike-timing-dependent plasticity (STDP) is a general learning rule in the brain, but spiking neural networks (SNNs) trained with STDP alone is inefficient and perform poorly. In this paper, taking inspiration from short-term synaptic plasticity, we design an adaptive synaptic filter and introduce the adaptive spiking threshold as the neuron plasticity to enrich the representation ability of SNNs. We also introduce an adaptive lateral inhibitory connection to adjust the spikes balance dynamically to help the network learn richer features. To speed up and stabilize the training of unsupervised spiking neural networks, we design a samples temporal batch STDP (STB-STDP), which updates weights based on multiple samples and moments. By integrating the above three adaptive mechanisms and STB-STDP, our model greatly accelerates the training of unsupervised spiking neural networks and improves the performance of unsupervised SNNs on complex tasks. Our model achieves the current state-of-the-art performance of unsupervised STDP-based SNNs in the MNIST and FashionMNIST datasets. Further, we tested on the more complex CIFAR10 dataset, and the results fully illustrate the superiority of our algorithm. Our model is also the first work to apply unsupervised STDP-based SNNs to CIFAR10. At the same time, in the small-sample learning scenario, it will far exceed the supervised ANN using the same structure. [ABSTRACT FROM AUTHOR]

Universal domain adaptation (UniDA) is an unsupervised domain adaptation that selectively transfers the knowledge between different domains containing different label sets. However, the existing methods do not predict the common labels of different domains and manually set a threshold to discriminate private samples, so they rely on the target domain to finely select the threshold and ignore the problem of negative transfer. In this paper, to address the above problems, we propose a novel classification model named Prediction of Common Labels (PCL) for UniDA, in which the common labels are predicted by Category Separation via Clustering (CSC). It is noted that we devise a new evaluation metric called category separation accuracy to measure the performance of category separation. To weaken negative transfer, we select source samples by the predicted common labels to fine-tune model for better domain alignment. In the test process, the target samples are discriminated by the predicted common labels and the results of clustering. Experimental results on three widely used benchmark datasets indicate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Supervisory Control and Data Acquisition (SCADA) systems are computer-based control architectures specifically engineered for the operation of industrial machinery via hardware and software models. These systems are used to project, monitor, and automate the state of the operational network through the utilization of ethernet links, which enable two-way communications. However, as a result of their constant connectivity to the internet and the lack of security frameworks within their internal architecture, they are susceptible to cyber-attacks. In light of this, we have proposed an intrusion detection algorithm, intending to alleviate this security bottleneck. The proposed algorithm, the Genetically Seeded Flora (GSF) feature optimization algorithm, is integrated with Transformer Neural Network (TNN) and functions by detecting changes in operational patterns that may be indicative of an intruder's involvement. The proposed Genetically Seeded Flora Transformer Neural Network (GSFTNN) algorithm stands in stark contrast to the signature-based method employed by traditional intrusion detection systems. To evaluate the performance of the proposed algorithm, extensive experiments are conducted using the WUSTL-IIOT-2018 ICS SCADA cyber security dataset. The results of these experiments indicate that the proposed algorithm outperforms traditional algorithms such as Residual Neural Networks (ResNet), Recurrent Neural Networks (RNN), and Long Short-Term Memory (LSTM) in terms of accuracy and efficiency. • The paper proposes a hybrid algorithm called the Genetically Seeded Flora Transformer Neural Network (GSFTNN) to detect intrusion in smart grid SCADA systems. • The performance of the proposed algorithm was evaluated using the WUSTL-IIOT-2018 SCADA cyber security dataset. • The proposed algorithm outperforms traditional intrusion detection systems such as ResNet, RNN, and LSTM in terms of accuracy and efficiency. [ABSTRACT FROM AUTHOR]

Few-shot learning aims to train a model with a limited number of base class samples to classify the novel class samples. However, to attain generalization with a limited number of samples is not a trivial task. This paper proposed a novel few-shot learning approach named Self-supervised Contrastive Learning (SCL) that enriched the model representation with multiple self-supervision objectives. Given the base class samples, the model is trained with the base class loss. Subsequently, contrastive-based self-supervision is introduced to minimize the distance between each training sample with their augmented variants to improve the sample discrimination. To recognize the distant sample, rotation-based self-supervision is proposed to enable the model to learn to recognize the rotation degree of the samples for better sample diversity. The multitask environment is introduced where each training sample is assigned with two class labels: base class label and rotation class label. Complex augmentation is put forth to help the model learn a deeper understanding of the object. The image structure of the training samples are augmented independent of the base class information. The proposed SCL is trained to minimize the base class loss, contrastive distance loss, and rotation class loss simultaneously to learn the generic features and improve the novel class performance. With the multiple self-supervision objectives, the proposed SCL outperforms state-of-the-art few-shot approaches on few-shot image classification benchmark datasets. • A novel self-supervised contrastive learning for few-shot image classification. • Contrastive learning is introduced to obtain a better sample discrimination. • Rotation prediction is proposed to enhance the sample diversity. • Heavy transformation is proposed for deeper object understanding. [ABSTRACT FROM AUTHOR]

Weight-tied models have attracted attention in the modern development of neural networks. The deep equilibrium model (DEQ) represents infinitely deep neural networks with weight-tying, and recent studies have shown the potential of this type of approach. DEQs are needed to iteratively solve root-finding problems in training and are built on the assumption that the underlying dynamics determined by the models converge to a fixed point. In this paper, we present the stable invariant model (SIM) , a new class of deep models that in principle approximates DEQs under stability and extends the dynamics to more general ones converging to an invariant set (not restricted in a fixed point). The key ingredient in deriving SIMs is a representation of the dynamics with the spectra of the Koopman and Perron–Frobenius operators. This perspective approximately reveals stable dynamics with DEQs and then derives two variants of SIMs. We also propose an implementation of SIMs that can be learned in the same way as feedforward models. We illustrate the empirical performance of SIMs with experiments and demonstrate that SIMs achieve comparative or superior performance against DEQs in several learning tasks. • A Koopman spectral analysis is performed for deep equilibrium models. • The analysis identifies stable dynamics determined by deep equilibrium models. • We develop a class of deep models to approximate the stable dynamics. • Models in the class can represent dynamics converging to an invariant set. [ABSTRACT FROM AUTHOR]

Deep learning (DL) applied to breast tissue segmentation in magnetic resonance imaging (MRI) has received increased attention in the last decade, however, the domain shift which arises from different vendors, acquisition protocols, and biological heterogeneity, remains an important but challenging obstacle on the path towards clinical implementation. In this paper, we propose a novel Multi-level Semantic-guided Contrastive Domain Adaptation (MSCDA) framework to address this issue in an unsupervised manner. Our approach incorporates self-training with contrastive learning to align feature representations between domains. In particular, we extend the contrastive loss by incorporating pixel-to-pixel, pixel-to-centroid, and centroid-to-centroid contrasts to better exploit the underlying semantic information of the image at different levels. To resolve the data imbalance problem, we utilize a category-wise cross-domain sampling strategy to sample anchors from target images and build a hybrid memory bank to store samples from source images. We have validated MSCDA with a challenging task of cross-domain breast MRI segmentation between datasets of healthy volunteers and invasive breast cancer patients. Extensive experiments show that MSCDA effectively improves the model's feature alignment capabilities between domains, outperforming state-of-the-art methods. Furthermore, the framework is shown to be label-efficient, achieving good performance with a smaller source dataset. The code is publicly available at https://github.com/ShengKuangCN/MSCDA. • Domain shift exists in breast MRI due to acquisition and biological heterogeneity. • Multi-level Semantic-guided Contrastive Domain Adaptation(MSCDA) has been proposed. • MSCDA uses different level of contrasts to exploit MRI semantic information. • MSCDA outperformed state-of-the-art methods in cross-sequence tasks. • Achieving similar performance on small source datasets shows MSCDA label-efficient. [ABSTRACT FROM AUTHOR]

• Temporal convolutional network with temporal position fusion for estimating evoked expressions in video. • Address the inconsistency of time steps during training and testing with temporal position fusion. • State-of-the-art result on Evoked Expressions from Videos (EEV) dataset. • Made huge improvements with temporal position fusion on MediaEval 2018 dataset. This paper introduces an approach for estimating evoked categories expression from videos with the temporal position fusion. Pre-trained models on large-scale datasets in computer vision and audio signals were used to extract the deep representation for timestamps in the video. A temporal convolution network, rather than an RNN-like architecture, was applied to explore temporal relationships due to its advantage in memory consumption and parallelism. Furthermore, to address the noise labels, the temporal position was fused with the deep learned feature to ensure the network differentiates the time steps when noise labels were removed from the training set. This technique helps the system gain a considerable improvement compared to other methods. We conducted experiments on EEV, a large-scale dataset for evoked expression from videos, and achieved a score of 0.054 in terms of Pearson correlation coefficient as a state-of-the-art result. Further experiments on a sub set of LIRIS-ACCEDE dataset - MediaEval 2018 benchmark, also demonstrated the effectiveness of our approach. [ABSTRACT FROM AUTHOR]

• A new deep metric learning loss function that enforces the neural network to learn a highly discriminative feature space. • An automated penalty-based loss function (APML) that ensures the best intra-class distance minimization. • A complementary loss function (MES) that explicitly implies a heavy constraint seeking to achieve an optimal inter-class separation. Deep metric learning has attracted much attention in recent years due to its extensive applications, such as clustering and image retrieval. Thanks to the success of deep learning (DL), many deep metric learning (DML) methods have been proposed. Neural networks (NNs) utilize DML loss functions to learn a mapping function that maps samples into a highly discriminative low-dimensional feature space, facilitating measuring similarities between pairs of samples in such a manifold. Most existing methods usually try to boost the discriminatory power of NN by enhancing intra-class compactness in the high-level feature space. However, they do not explicitly imply constraints to improve inter-class separation. We propose in this paper a new composite DML loss function that, in addition to the intra-class compactness, explicitly implies regulations to enforce the best inter-class separation by mutually equidistantly distributing the centers of the classes. The proposed DML loss function achieved state-of-the-art results for clustering and image-retrieval tasks on two real-world data sets. [ABSTRACT FROM AUTHOR]

• Online tracking and real-time multi-object tracking and segmentation(MOTS) tracker. • Mask-guided re-identification(ReID) using bounding box information and mask together. • End-to-end learning combining ReID branch with instance segmentation network. • Evaluation of an experiment in comparison with state-ofthe-art methods. • Ablation study to validate mask-guided ReID branch and feature dimension. Applications of multi-object tracking and segmentation (MOTS) include autonomous driving and video surveillance systems. In these applications, the tracking system must be able to track in real-time and online. Recent studies to improve the tracking performance of MOTS have been actively conducted; however, most studies fail to consider online and real-time performance. This paper proposes a MOTS system that operates in real time using a deep learning–based model with a light backbone, and can be tracked online. Additionally, state-of-the-art trackers use an object's bounding box to extract re-identification (ReID) features, but include unnecessary background. Additionally, this causes confusion in accurately expressing the appearance features of an object, resulting in difficulty in matching. To solve this difficulty, instead of providing the features of the bounding box as the input of the ReID branch, we focus on expressing the object by providing the mask features of the object as an input. As a result of the mask-based ReID experiment, the association accuracy performance was higher than that of the existing bounding box–based ReID model, and among the KITTI MOTS benchmarks, it ranked second among models that can operate online. Our experiments show that background information causes ambiguous ReID matching in MOTS systems, and that object mask information is important for avoiding ambiguous matching. [ABSTRACT FROM AUTHOR]

• An end-to-end one stage detection model to detect multiscale and rotated objects through aerial image. • Performing both the tasks, i.e., horizontal bounding box detection and oriented bounding box detection for input image. • Identifying aerial image objects with high inference speed and detection mean Average Precision (mAP) accuracy. • Ensuring the robustness for adversarial attacks such as FGSM, BIM, and JSM attacks. Object recognition and localization play a crucial role in aerial images and their applications. The aerial images are challenging due to the large aspect ratio, arbitrary orientation, variation in scales, and non-uniform and cluttered object distribution. To address these challenges, we propose an efficient and robust model called the Simultaneous Object Localization and Recognition Network (SOLARNet), which is a fusion network that integrates two different sub-networks: PixelAttentionDetector (PD) and RotationDetector (RD). The PD considers features from different scales and cluttered objects, while RD handles rotation invariance, giving horizontal and oriented object detection results. The state-of-the-art model fails to improve accuracy when images are adversarially attacked. SOLARNet is not only efficient in terms of accuracy but also robust concerning to Fast Gradient Sign Method (FGSM), Basic Iterative Method (BIM), Jacobian Based Saliency Map (JSM) adversarial attacks, which is the crucial factor for any mission-critical system. We have performed experiments and achieved the accuracy on the publicly available DOTA dataset (75.00% mAP, 66.40% mAP) and DIOR dataset (88.60% mAP, 81.50% mAP) for horizontal and oriented object recognition tasks respectively while having high inference speed. Qualitative and quantitative results reported in this paper substantiate the superiority of SOLARNet over other state-of-the-art methodologies. [ABSTRACT FROM AUTHOR]

• Develop a dataset of Mars scenes from numerous raw Mars images. • Compare performance of three common deep learning models to select the appropriate one. • Conduct transfer learning and data augmentation to enhance selected model's generation capacity. • Deep learning can realize autonomous, real-time Martian terrain classification. Scene information plays a crucial role in motion control, attitude perception, and path planning for wheeled planetary rovers (WPRs). Terrain recognition is the fundamental component of scene recognition. Due to the rich information, visual sensors are usually used in terrain classification. However, teleoperation delay prevents WPRs from using visual information efficiently. End-to-end learning method of deep learning (DL) that does not need complex image preprocessing was proposed to deal with this issue. This paper first built a terrain dataset (consists of loose sand, bedrock, small rock, large rock, and outcrop) using real Mars images to directly support You Only Look Once (YOLOv5) to test its performance on terrain classification. Because the capability of end-to-end training scheme is positively correlated with dataset, the performance of YOLOv5 can be significantly improved by exploiting orders of magnitude more data. The best combination of hyperparameters and models was achieved by slightly tuning YOLOv5, and data augmentation was also applied to optimize its accuracy. Furthermore, its performance was compared with two other end-to-end network architectures. Deep learning algorithms can be used in the future planetary exploration missions, such as WPRs autonomy improvement, traversability analysis, and avoiding getting trapped. [ABSTRACT FROM AUTHOR]

Identification of human actions from video has gathered much attention in past few years. Most of the computer vision tasks such as Health Care Activity Detection, Suspicious Activity detection, Human Computer Interactions etc. are based on the principle of activity detection. Automatic labelling of activity from videos frames is known as activity detection. With the introduction of deep networks, the process of activity detection is clustered into two groups known as hand-crafted feature based approach and automatic feature extraction approach. This paper focuses on various approaches used in recent literature based on traditional and automatic approach. Moreover, hierarchy for different approaches under them such as space based, motion based, genetic based, fuzzy based, dictionary based are discussed. With introduction of Convolutional Neural Networks and Recurrent Neural Networks, automatic learning capability from input modality makes them first choice to be implemented for activity recognition. In this paper various approaches have been analyzed according to methodology, accuracy, classifier and datasets. [ABSTRACT FROM AUTHOR]

In today's email dependent world, cyber criminals often target organizations using a variety of social engineering techniques and specially crafted malicious emails. When successful, such attacks can result in significant harm to physical and digital systems and assets, the leakage of sensitive information, reputation damage, and financial loss. Despite the plethora of studies on the detection of phishing attacks and malicious links in emails, there are no solutions capable of effectively, quickly, and accurately coping with more complex email-based attacks, such as malicious email attachments. This paper presents the first fully automated malicious email detection framework using deep ensemble learning to analyze all email segments (body, header, and attachments); this eliminates the need for human expert intervention for feature engineering. In this paper, we also demonstrate how an ensemble framework of deep learning classifiers each of which are trained on specific portions of an email (thereby independently utilizing the entire email) can generalize better than popular email analysis methods that analyze just a specific portion of the email for analysis. The proposed framework is evaluated comprehensively and with an AUC of 0.993, the proposed framework's results surpass state-of-the-art malicious email detection methods, including human expert feature-based machine learning models by a TPR of 5%. [ABSTRACT FROM AUTHOR]

• For the first time, AUs and optical flow features are combined to spot either macro- or micro- expression intervals. • The proposal can eliminate the influence of facial image change caused by noises, such as body or head movement. • The proposed Concat-CNN model can learn both the inner features of a single frame and the correlation between frames. • The re-labeling method considers the overall change process of a specific expression and improves the detection performance. • The proposal shows remarkable improvement in the F1 scores on datasets, such as the CAS(ME) 2 -cropped and the SAMM-LV. [Display omitted] This paper is an extension of our previously published ACM Multimedia 2022 paper, which was ranked 3rd in the macro-expressions (MaEs) and micro-expressions (MEs) spotting task of the FME challenge 2021. In our earlier work, a deep learning framework based on facial action units (AUs) was proposed to emphasize both local and global features to deal with the MaEs and MEs spotting tasks. In this paper, an advanced Concat-CNN model is proposed to not only utilize facial action units (AU) features, which our previous work proved were more effective in detecting MaEs, but also to fuse the optical flow features to improve the detection performance of MEs. The advanced Concat-CNN proposed in this paper not only considers the intra-features correlation of a single frame but also the inter-features correlation between frames. Further, we devise a new adaptive re-labeling method by labeling the emotional frames with distinctive scores. This method takes into account the dynamic changes in expressions to further improve the overall detection performance. Compared with our earlier work and several existing works, the newly proposed deep learning pipeline is able to achieve a better performance in terms of the overall F1-scores: 0.2623 on CAS(ME) 2 , 0.2839 on CAS(ME) 2 -cropped, and 0.3241 on SAMM-LV, respectively. [ABSTRACT FROM AUTHOR]

Solar flares are a kind of violent solar eruptive activity phenomenon and an important warning device of space weather disturbance. In space weather forecasting, flare forecasting is an important forecast content. This paper proposes a flare prediction model based on long and short-term memory neural network, which uses the time sequence of magnetic field changes in the solar active region in the past 24 h to construct samples, and analyzes the time series evolution of magnetic field characteristics through the long and short-term memory neural network to predict whether ≥ M-level flares will occur in the next 48 h. This paper uses a data set for all active region samples from May 2010 to May 2017, and selects 10 magnetic field characteristic parameters of SDO/HMI SHARP. In the modeling process, six feature parameters with high weight, gain rate, and coverage rate were selected as input parameters through XGBoost method. Through test comparison, the false report rate and accuracy rate of the model are similar to the traditional machine learning model, and the accuracy rate and critical success index are better than the traditional machine learning model, which are 0.7483 and 0.7402, respectively. The overall effect of the model is better than that of the traditional machine learning model. [ABSTRACT FROM AUTHOR]

• This paper proposes a filter pruning approach for CNN model compression by transferring the knowledge of unimportant filters to the filters of higher importance. • Before pruning unimportant filters, a custom regularizer is utilized for knowledge transfer, which increases the gap between L1-norms of important and unimportant filters. • The effect of the penalty imposed in the custom regularizer is analyzed to justify the need for knowledge transfer before pruning. • In order to validate the robustness of the proposed framework across different CNN architectures, we experiment with five popular CNNs, namely, LeNet-5, VGG-16, ResNet-56, ResNet-110, and ResNet-50. • Experiments are performed on three benchmark datasets MNIST, CIFAR-10, and ImageNet. • An improvement over the baseline in terms of accuracy is observed even after removing 95.15%, 62.28%, and 62.39% of the Floating Point OPerations (FLOPs) from architectures LeNet-5, ResNet-56, and ResNet-110, respectively. As the deep learning models have been widely used in recent years, there is a high demand for reducing the model size in terms of memory and computation without much compromise in the model performance. Filter pruning is a very widely adopted strategy for model compression. The existing filter pruning methods identify the unimportant filters and prune them without worrying about information loss. They try to recover the same by fine-tuning the remaining filters, limiting their performance. In this paper, we tackle this problem by utilizing the knowledge from unimportant filters before pruning to minimize information loss. First, the proposed method identifies the unimportant and important filters by exploiting the lower and higher importance, respectively, using the L 1 -norm of filters. Next, the proposed custom UFKT-Reg regularizer ( R ufkt ) transfers the knowledge from unimportant filters before pruning to remaining filters, notably to a fixed number of important filters. Hence, the proposed method minimizes information loss due to the removal of unimportant filters. The experiments are conducted using the three benchmark datasets, including MNIST, CIFAR-10, and ImageNet. The proposed filter pruning method outperforms many recent state-of-the-art filter pruning methods. An improvement over the baseline in terms of accuracy is observed even after removing 95.15%, 62.28%, and 62.39% of the Floating Point OPerations (FLOPs) from architectures LeNet-5, ResNet-56, and ResNet-110, respectively. After pruning 53.25% of FLOPS from ResNet-50, only 1.02% and 0.47% of drops are observed in top-1 and top-5 accuracies, respectively. The code used in this paper will be publicly available at (https://github.com/sarvanichinthapalli/UFKT). [ABSTRACT FROM AUTHOR]

Modeling of large and complex industrial processes such as chemical processes has always been an ongoing research topic. Hybrid modeling approaches draw more and more attentions due to its ability to combine the benefits of both mechanism(first-principle) models and data-based models in the term of the balance between accuracy and cost. However, there is an insurmountable problem in the application of hybrid modeling, that is, when the accurate formulated mechanism information of the system is difficult to obtain, the hybrid modeling strategy is often difficult to obtain the desired results in existing hybrid modeling strategies. In order to solve the contradiction between the need for accurate formulated mechanism and the reality that obtaining them in a large actual process is difficult or even impossible, we utilized the directed graph and the method of Bayesian Neural Network(BNN)-Structure Learning to combine the mechanism information and data information into a hybrid information graph, which is called BNN-Data Augmented Graph(BNN-DAG) to form a hybrid model without accurate first-principles formulas. On the basis of the hybrid information graph, the system is first partitioned into small blocks according to the demand of the target variables, then an appropriate modeling method is utilized based on the requirements of the object. Then, the selected modeling method is implemented into blocks and contributes to a complete predictive model, so that the complex system can still be modeled with a certain accuracy without the mechanism information expressed by accurate formulas. The method in this manuscript is validated on a Tennessee Eastman(TE) benchmark process and a Hydrogenation Fractionation Unit(HFU) in a real plant. The prediction results are compared with the currently popular deep learning methods. It is proved that under the framework of the method in this manuscript, it is possible to obtain a system prediction model with the same or higher accuracy as the existing deep learning methods but with a relatively a low cost. Under the framework, the interpretability of the prediction model could be obtained clearly. Also, the improvement is beneficial to the application in the actual industrial process. • This paper proposes an enhanced Bayesian neural network structure learning strategy to address the hybrid modeling of complex systems without formalized mechanistic information. • In order to solve the problem of representing the non-formulated mechanism information of complex systems, directed graph networks are introduced to represent the mechanism information of the system. • Different from the fixed model structure in the previous hybrid modeling strategy, the model structure in the strategy proposed in this paper can be adjusted according to actual requirements, so as to obtain a reasonable balance of model complexity and accuracy. • A real chemical plant is used to verify the method proposed in this paper. The results show that the method can guarantee a certain accuracy and reduce the computational burden. [ABSTRACT FROM AUTHOR]

• An audio dataset with FSL and OSR considerations is proposed. • It contains real recordings from 34 classes of alarms and common domestic sounds. • Recordings were captured in real environments using the same device. • This dataset will avoid manipulations on existing datasets (common practice today). • Two baselines are proposed based on a transfer learning approach as starting points. The problem of training with a small set of positive samples is known as few-shot learning (FSL). It is widely known that traditional deep learning algorithms usually show very good performance when trained with large datasets. However, in many applications, it is not possible to obtain such a high number of samples. This paper deals with the application of FSL to the detection of specific and intentional acoustic events given by different types of sound alarms, such as door bells or fire alarms, using a limited number of samples. These sounds typically occur in domestic environments where many events corresponding to a wide variety of sound classes take place. Therefore, the detection of such alarms in a practical scenario can be considered an open-set recognition (OSR) problem. To address the lack of a dedicated public dataset for audio FSL, researchers usually make modifications on other available datasets. This paper is aimed at providing the audio recognition community with a carefully annotated dataset 1 for FSL in an OSR context comprised of 1360 clips from 34 classes divided into pattern sounds and unwanted sounds. To facilitate and promote research on this area, results with state-of-the-art baseline systems based on transfer learning are also presented. [ABSTRACT FROM AUTHOR]

• Solving the time-series anomaly detection problem in CPSSs by deep learning approach. • Combining the advantages of LSTM and reconstruction models for time- series anomaly detection problems. • Proposing the adversarial training method to improve the robustness of encoder-decoder model. • Avoiding the heavy workload of traditional methods for a priori knowledge and quality labeling. • Conducting the extensive experiments to prove the validity of the method in this paper. With the development and maturity of smart cities, more and more Cyber-Physical-Social Systems (CPSSs) need to monitor a variety of time-series data from sensors and network transmissions to ensure the quality and reliability of the Cyber-Physical-Social Services. Time-series anomaly detection is a common search problem in the field of pattern recognition. Existing approaches and models of anomaly detection have solved the problem of simple smooth time-series and perform ideal recognition performance. However, in real scenarios, complex time-series with non-Gaussian noise and complex data distributions are prevalent. Compared to smooth and simple time-series, complex time-series occur more frequently in real-world settings and are difficult to model and label. To address these challenges, this paper proposes an unsupervised anomaly detection algorithm based on Long Short-Term Memory Encoder-Decoder (LSTM-ED) via an adversarial training method for complex time-series in cyber-physical-social systems with high performance. This is a novel method that incorporates adversarial learning to improve the robustness of encoder-decoder architecture, enabling it to obtain good anomaly detection results for complex time-series. In addition, LSTM is employed as the network unit of encoder-decoder architecture, which is also able to extract temporal correlation in time-series to a greater extent. We have conducted extensive experiments on four datasets from real scenarios and the results show that the accuracy of the proposed adversarial training of LSTM-ED is significantly better than that of the state-of-the-art methods, including other unsupervised methods and traditional supervised methods. [ABSTRACT FROM AUTHOR]

In this paper, an approach that makes use of knuckle creases and fingernails for person identification is presented. It introduces a framework for automatic person identification that includes localisation of the region of interest (ROI) of many components within hand images, recognition and segmentation of the detected components using bounding boxes, and similarity matching between two different sets of segmented images. The following hand components are considered: i) the metacarpophalangeal (MCP) joint, commonly known as the base knuckle; ii) the proximal interphalangeal (PIP) joint, commonly known as the major knuckle; iii) the distal interphalangeal (DIP) joint, commonly known as the minor knuckle; iv) the interphalangeal (IP) joint, commonly known as the thumb knuckle, and v) the fingernails. Crucial elements of the proposed framework are the feature extraction and similarity matching. This paper exploits different deep learning neural networks (DLNNs), which are essential in extracting discriminative high-level abstract features. We further use various similarity measures for the matching process. We validate the proposed approach on well-known benchmarks, including the 11k Hands dataset and the Hong Kong Polytechnic University Contactless Hand Dorsal Images known as PolyU. The results indicate that knuckle patterns and fingernails play a significant role in the person identification framework. The 11K Hands dataset results indicate that the left-hand results are better than the right-hand results and the fingernails produce consistently higher identification results than other hand components, with a rank-1 score of 100 %. In addition, the PolyU dataset attains 100 % in the fingernail of the thumb finger. [ABSTRACT FROM AUTHOR]

In current deep learning architectures, each of the deeper layers in networks tends to contain hundreds of unorganized neurons which makes it hard for humans to understand how they interact with each other. By organizing the neurons using correlation as the criteria, humans can observe how clusters of neighbouring neurons interact with each other. Research in Explainable Artificial Intelligence (XAI) aims to all alleviate the black-box nature of current AI methods and make them understandable by humans. In this paper, we extend our previous algorithm for XAI in deep learning, called Locality Guided Neural Network (LGNN). LGNN preserves locality between neighbouring neurons within each layer of a deep network during training. Motivated by Self-Organizing Maps (SOMs), the goal is to enforce a local topology on each layer of a deep network such that neighbouring neurons are highly correlated with each other. Our algorithm can be easily plugged into current state of the art Convolutional Neural Network (CNN) models to make the neighbouring neurons more correlated. A cluster of neighbouring neurons activating for a class makes the network both quantitatively and qualitatively more interpretable when visualized, as we show through our experiments. This paper focuses on image processing with CNNs, but can theoretically be applied to any type of deep learning architecture. In our experiments, we train VGG and WRN networks for image classification on CIFAR100 and Imagenette. Our experiments analyse different perceptible clusters of activations in response to different input classes. [ABSTRACT FROM AUTHOR]

• The paper fills the gap in the literature by reviewing and analyzing the current studies that apply deep learning for Covid-19 forecasting. • The initial search identified 152 studies of which 53 passed the quality control. • The existing literature is categorized using a model-based taxonomy. • The description of the models along with their performance evaluation is presented. • Recommendations for future improvements are provided. The Covid-19 pandemic has galvanized scientists to apply machine learning methods to help combat the crisis. Despite the significant amount of research there exists no comprehensive survey devoted specifically to examining deep learning methods for Covid-19 forecasting. In this paper, we fill the gap in the literature by reviewing and analyzing the current studies that use deep learning for Covid-19 forecasting. In our review, all published papers and preprints, discoverable through Google Scholar, for the period from Apr 1, 2020 to Feb 20, 2022 which describe deep learning approaches to forecasting Covid-19 were considered. Our search identified 152 studies, of which 53 passed the initial quality screening and were included in our survey. We propose a model-based taxonomy to categorize the literature. We describe each model and highlight its performance. Finally, the deficiencies of the existing approaches are identified and the necessary improvements for future research are elucidated. The study provides a gateway for researchers who are interested in forecasting Covid-19 using deep learning. [ABSTRACT FROM AUTHOR]

Search and retrieval remains a major research topic in several domains, including computer graphics, computer vision, engineering design, etc. A search engine requires primarily an input search query and a database of items to search from. In engineering, which is the primary context of this paper, the database consists of 3D CAD models, such as washers, pistons, connecting rods, etc. A query from a user is typically in the form of a sketch, which attempts to capture the details of a 3D model. However, sketches have certain typical defects such as gaps, over-drawn portions (multi-strokes), etc. Since the retrieved results are only as good as the input query, sketches need cleaning-up and enhancement for better retrieval results. In this paper, a deep learning approach is proposed to improve or clean the query sketches. Initially, sketches from various categories are analysed in order to understand the many possible defects that may occur. A dataset of cleaned-up or enhanced query sketches is then created based on an understanding of these defects. Consequently, an end-to-end training of a deep neural network is carried out in order to provide a mapping between the defective and the clean sketches. This network takes the defective query sketch as the input and generates a clean or an enhanced query sketch. Qualitative and quantitative comparisons of the proposed approach with other state-of-the-art techniques show that the proposed approach is effective. The results of the search engine are reported using both the defective and enhanced query sketches, and it is shown that using the enhanced query sketches from the developed approach yields improved search results. [Display omitted] • The first learning-based strategy to clean rough query sketches of 3D CAD models • Introduces SketchCleanNet — an end-to-end image translation scheme • SketchCleanNet aims to understand the mapping between rough sketches and clean query images • A novel scheme to calculate the loss is introduced • Dataset Contribution: The resulting enhanced query sketch dataset is made available publicly. • This paper will significantly contribute to the research community and give researchers opportunities to develop new algorithms for search and retrieval of 3D mechanical components. [ABSTRACT FROM AUTHOR]

Concentrated Solar Power plants (CSP) have the energy storage capability to generate electricity when sunlight is scarce. However, due to the highly non-linear dynamics of these systems, a simple linear controller will not be able to overcome the variable dynamics and multiple disturbance sources affecting it. In this paper, a deep Model Predictive Control (MPC) based on the Koopman operator is proposed and applied to control the Heat Transfer Fluid (HTF) temperature of a distributed-parameter model of the ACUREX solar collector field located at Almería, Spain. The Koopman operator is an infinite-dimensional linear operator that fully captures a system's non-linear dynamics through the linear evolution of functions of the state-space. However, one of the major problems is identifying a Koopman linear model for a non-linear system. Koopman eigenfunctions are involved in converting a non-linear model to a Koopman-based linear model. In this paper, a deep Long Short-Term Memory (LSTM) autoencoder is designed to calculate Koopman eigenfunctions of the solar collector field. The Koopman linear model is then used to design a linear MPC with terminal components to ensure closed-loop stability guarantees. Simulation results are utilized to show the satisfactory tracking performance of the proposed approach. [ABSTRACT FROM AUTHOR]

• This paper proposes a re-identification method based on visual common attention mechanism • This paper proposes a joint feature method for pedestrian re-identification • This paper proposes a new strategy for generating difficult positive samples. In view of the strong generalizability and self-learning capabilities of deep learning models, many scholars have studies how to apply deep learning theory in the pedestrian re-identification field. However, a number of problems persist in practically applying deep learning in this field, including determining how to make full use of the features of the sequence information in the salient region of an image and addressing the data gap between data-driven deep learning models and pedestrian re-identification tasks. In view of these problems, in this paper, a re-identification method is proposed based on a visual common attention mechanism. Initially, the method focuses on the local area of the image at the location specified by the given coordinates. Next, under the constraint of pedestrian image pairing tags, it focuses on the sequence of salient regions of image pairs based on deep learning techniques. Then, the global features and the local attention features are cascaded into joint features for use in pedestrian re-identification. To address the data gap between deep learning models and pedestrian re-identification, a new strategy for generating difficult positive samples is proposed primarily through a positive sample that mainly involves a positive sample generation network, a difficult positive sample conversion network, and a dual-stream twin network. We using this network, a large number of positive samples can be obtained to train the data-driven neural network and solve the re-identification task. The above ideas are combined to propose a pedestrian re-identification algorithm based on a visual attention-positive sample generation network deep learning model. The experimental results show that the method proposed in this paper not only achieves better recognition results than other deep learning methods, but also adapts well to a variety of databases. In addition, the method proposed in this paper is more robust to occluded pedestrian images than other deep learning methods. In addition, the method proposed in this paper is more robust than other deep learning methods for occluding pedestrian images. [ABSTRACT FROM AUTHOR]

• An overview of deep learning algorithms used in medical image segmentation is presented. • More than 150 papers applying deep learning to different medical applications are summarised. • Challenges and future directions in medical image segmentation are discussed. Deep learning (DL) algorithms have rapidly become a robust tool for analyzing medical images. They have been used extensively for medical image segmentation as the first and significant components of the diagnosis and treatment pipeline. Medical image segmentation is efficiently addressed by many types of deep neural networks, such as convolutional neural networks, fully convolutional network recurrent networks, adversarial networks, and U-shaped networks. This paper reviews the major DL models and applications pertinent to medical image segmentation and summarizes over 150 contributions to the field. Brief overviews of articles are provided by application area: anatomical structures such as organs, bones, and vessels, and abnormalities such as lesions and calcification. Moreover, we discuss current challenges and suggest directions for future research. [ABSTRACT FROM AUTHOR]

• A detailed literature review for the existing methods about anchor assignment and sample sampling is investigated and analyzed in a taxonomy. We identify a definition of each component, challenging problems in existing methods and their solution, and a comprehensive comparison of all methods. To the best of our knowledge, there is no prior work discussing these problems in object detection literature. • Open research direction in both anchor assignment and sampling heuristics is discussed. Based on these directions, we hope the object detection community can identify the current status of object detection methods to propose better solutions to anchor assignment and sampling heuristics. • We review the state-of-the-art object detection in the last two years that complement existing surveys. We also provide paper literature related to the above problems via the repository webpage. Deep learning-based object detection is a fundamental but challenging problem in computer vision field, has attracted a lot of study in recent years. State-of-the-art object detection methods rely on the selection of positive samples and negative samples, i.e., called sample assignment, and the definition of a useful set for training, i.e., called sample sampling heuristics. This paper presents a comprehensive review of the advanced anchor assignment and sampling approaches in deep learning-based object detection. Each problem is classified and analyzed systematically. According to the problem-based taxonomy, we identify the advantages and disadvantages of each problem in-depth and present open issues regarding the current methods. Furthermore, this paper also reviews the new trends in solving object detection that has not been discussed during the last two years. To track the latest research, a webpage related to the above problems is provided, which is available at https://github.com/VoXuanThuy/ObjectDetectionReview. [ABSTRACT FROM AUTHOR]

• Model-based and model-free gait recognition techniques apt for handling covariates. • An exploration of benefits/ limitations of deep learning pipelines under covariates. • List of covariates-based gait datasets for training robust gait recognition system. • Overview of the proposed fusion-based pipeline with the 3D dataset for covariates. • Research gaps and challenges in the implementation of deep learning pipelines. Gait recognition is an essential biometric technique that recognizes humans at a distance through their unique walking style. In the present era of deep learning, automated gait analysis has become easier with an increase in processing power. However, the recognition accuracy is affected by many covariates such as clothing conditions, carrying objects, varying viewing angles, occlusion, walking speed variations, and thus, it remains a challenging problem. For this complex problem, huge datasets are required to train for given conditions and predict new situations; thus, deep learning is preferred. In this review paper, we categorize various gait covariates which have been recently handled. There are various approaches, but the most effective approach is deep learning; hence in this paper, we include the most used deep learning approaches for each covariate condition found in the literature. Further, we highlight open problem areas handling these covariates and offer some suggestions about their better handling. Based on the review and our understanding of all the gait pipelines employed in deep learning, we have suggested a comprehensive and universal deep learning pipeline that can handle most gait covariates rather than customized deep learning pipelines. The methods of handling gait covariates are summarized according to appearance, pose, and sensors. A comprehensive comparison of reviewed approaches for real-time scenarios in terms of their novelty, benefits, and limitations is then offered, which led us to identify open research problems related to gait covariates. In the end, the paper concludes with the challenges identified and prospects. [ABSTRACT FROM AUTHOR]

Foreign species can deteriorate the environment and the economy of a country. To automatically monitor biosecurity threats at country borders, this paper investigates compact deep networks for accurate and real-time object segmentation for hyperspectral images. To this end, knowledge distillation (KD) approaches compress the model by distilling the knowledge of a large teacher network to a compact student network. However, when the student is over-compressed, the performance of standard KD methods degrades significantly due to the large capacity gap between the teacher and the student. This gap can be addressed by adding medium-sized teacher assistants, but training them incurs significant computation and hence is impractical. To address this problem, this paper proposes a new framework called Knowledge Distillation from Multi-head Teacher (KDM), which distills the knowledge of a multi-head teacher to the student. By encapsulating multiple teachers in a single network, our proposed KDM assists the learning of a very compact student and significantly reduces the training time. We also introduce Bio-HSI, a new large benchmark hyperspectral image dataset of 3,125 high-resolution images with dense segmentation ground truth. This new, large dataset can be expected to advance research on deep models for hyperspectral image segmentation. Evaluated on this dataset, the student trained via our KDM has 762 times fewer parameters than the state-of-the-art segmentation model (i.e., HRNet), while achieving competitive accuracy. [ABSTRACT FROM AUTHOR]

The problem of global warming is getting more serious, and vehicle emission is the main cause. In recent years, the number of locomotives in China has been increasing at a rate of more than 20% per year, and the problem of automobile pollution is becoming more serious. The transportation industry is the main source of fossil fuel combustion and environmental pollution. Therefore, in this paper, we propose a multi-view deep neural network (MVDNN) to analyze the key factors affecting the fuel consumption of automobiles. The experiments show that the introduction of human input improves the prediction accuracy and the root mean square error (RMSE) achieves 0.993. In addition, this paper also finds that for drivers, driving habits, driving frequency, and safety awareness are the most important factors affecting the fuel consumption of vehicles by combining Lasso regression with MVDNN. Finally, by comparing the prediction accuracy of different experiments, relevant policy suggestions are made. [ABSTRACT FROM AUTHOR]

Two participants with competing objectives are considered in one-to-one orbital pursuit-evasion problem, one of which attempts to satisfy some certain terminal conditions with a minimum cost through its own control while the other player attempts to maximize the specified payoff and avoids entering this terminal range. This paper treats the problem of minimax time to a given final distance, within which the interception between pursuer and evader is defined to occur. To achieve the successful interception with an evader that adopts any unknown maneuvers, a near-optimal interception strategy for the orbital pursuit-evasion is presented in this paper. The pursuit-evasion problem is formulated as a differential game first, and a closed-form barrier solution is provided next. A near-optimal guidance law using deep learning is proposed to intercept the evader inside the capture zone. For the games that start outside the barrier, a learning algorithm for the capture zone embedding strategy is presented based on deep reinforcement learning to help the game state cross the barrier surfaces. The simulation cases of intercepting the evaders that adopt different maneuvering strategies are provided, and the results demonstrate the effectiveness and onboard feasibility of the proposed strategy in solving the orbital interception problem. • The closed-form barrier solution for a planar orbital pursuit-evasion game is provided. • A deep neural network is trained on a dataset consisting of plenty of barrier trajectories generated through the closed-form barrier formula. • The near-optimal guidance law inside capture zone is proposed using trained neural network. • A capture zone embedding strategy is presented based on deep reinforcement learning. • The closed loop interception strategy is proposed combining the strategies outside and inside capture zone. [ABSTRACT FROM AUTHOR]

Computational Ethology studies focused on human beings is usually referred as Human Activity Recognition (HAR). Specifically, this paper belongs to a line of work on the identification of broad cognitive activities that users carry out with computers. The keystone of this kind of systems is the noninvasive detection of the subject's gaze fixations in selected display areas. Noninvasiveness is ensured by using the conventional laptop cameras without additional illumination or tracking devices. The gaze ethograms, composed as sequences of gaze fixations, are the basis to identify the user activities. To determine the gaze fixation display areas with the highest accuracy, this paper explores the use of a transfer learning approach applied to several well-known deep learning network (DLN) architectures whose input is the eye area extracted from the face image,and output is the identification of the gaze fixation area in the computer screen. Two different datasets are created and used in the validation experiments. We report encouraging results that may allow the general use of the system. [ABSTRACT FROM AUTHOR]