Search

Machine learning in medical imaging often faces a fundamental dilemma, namely, the small sample size problem. Many recent studies suggest using multi-domain data pooled from different acquisition sites/centers to improve statistical power. However, medical images from different sites cannot be easily shared to build large datasets for model training due to privacy protection reasons. As a promising solution, federated learning, which enables collaborative training of machine learning models based on data from different sites without cross-site data sharing, has attracted considerable attention recently. In this paper, we conduct a comprehensive survey of the recent development of federated learning methods in medical image analysis. We have systematically gathered research papers on federated learning and its applications in medical image analysis published between 2017 and 2023. Our search and compilation were conducted using databases from IEEE Xplore, ACM Digital Library, Science Direct, Springer Link, Web of Science, Google Scholar, and PubMed. In this survey, we first introduce the background of federated learning for dealing with privacy protection and collaborative learning issues. We then present a comprehensive review of recent advances in federated learning methods for medical image analysis. Specifically, existing methods are categorized based on three critical aspects of a federated learning system, including client end, server end, and communication techniques. In each category, we summarize the existing federated learning methods according to specific research problems in medical image analysis and also provide insights into the motivations of different approaches. In addition, we provide a review of existing benchmark medical imaging datasets and software platforms for current federated learning research. We also conduct an experimental study to empirically evaluate typical federated learning methods for medical image analysis. This survey can help to better understand the current research status, challenges, and potential research opportunities in this promising research field. • Summarize existing methods from a system perspective. • Introduce different methods in a "question–answer" paradigm. • Introduce software platforms and benchmark datasets. • Conduct an experimental study. [ABSTRACT FROM AUTHOR]

Over the past years, 3D point cloud registration has attracted unprecedented attention. Researchers develop various approaches to tackle the challenging task, such as optimization-based and deep learning-based methods. To systematically sort out the relevant literature and follow the state-of-the-art solutions, this paper conducts a thorough survey. We propose a novel taxonomy dubbed Intermediates Based Taxon (IBTaxon) which effectively categorizes multifarious registration approaches by the introduced intermediate variables or the leveraged intermediate modules. We further delve into each of the categories and present a comprehensive technique review with a focus on the distinct insight behind each of the methods. Besides, the relevant datasets and evaluation metrics are also combed and reorganized. We conclude our paper by discussing the possible open research problems and presenting our visions for future research in the field of 3D point cloud registration. • A novel taxonomy dubbed IBTaxon is proposed which categorizes registration methods as one-stage and two-stage approaches. • Following the IBTaxon, alternate and sequential optimization are induced as two strategies for two-stage approaches to achieve alignment. • Several widely used datasets and metrics for the standard evaluation and comparison of various registration methods are combed. • Experimental performances of several representative methods have been arranged and analyzed to provide a reference. • We make the conclusion that the balance between accuracy, speed, and robustness is more considerable rather than aspiring to a single indicator. [ABSTRACT FROM AUTHOR]

Bridging visual and textual representations plays a central role in delving into multimedia data understanding. The main challenge arises from that images and texts exist in heterogeneous spaces, leading to the difficulty to preserve the semantic consistency between both modalities. To narrow the modality gap, most recent methods resort to extra object detectors or parsers to obtain the hierarchical representations. In this work, we address this problem by introducing our Multi-Task Hierarchical Convolutional Neural Network (MT-HCN). It is characterized by mining the hierarchical semantic information without the aid of any extra supervisions. Firstly, from the perspective of representing architecture, we leverage the intrinsic hierarchical structure of Convolutional Neural Networks (CNNs) to decompose the representations of both modalities into two semantically complementary levels, i.e. , exterior representations and concept representations. The former focuses on discovering the fine-grained low-level associations between both modalities, meanwhile the latter underlines capturing more high-level abstract semantics. Specifically, we present a Self-Supervised Clustering (SSC) loss to preserve more fine-grained semantic clues in exterior representations. It is constituted on the basis of viewing multiple image/text pairs with similar exterior as a category. In addition, a novel harmonious bidirectional triplet ranking (HBTR) loss is proposed, which mitigate the adverse effects brought about by the biased and noisy negative samples. Besides hardest negatives, it also imposes the constraints on the distance between the positive pairs and the centroid of negative pairs. Extensive experimental results on two popular cross-modal retrieval benchmarks demonstrate our proposed MT-HCN can achieve the competitive results compared with the state-of-the-art methods. • This paper proposes a novel Multi-Task Hierarchical Convolutional Network (MT-HCN) for visual-semantic cross-modal retrieval, which is characterized by adopting classification task to improve the hierarchical multi-modal representation learning. • This paper proposes a novel Self-Supervision Clustering (SSC) loss to learn the exterior representations that fully exploits low-level fine-grained correlation for associating images and texts. • This paper presents an effective bidirectional ranking loss, namely Harmonious Bidirectional Ranking (HBR) for cross-modal correlation preserving. It not only efficiently assists us to seek out more representative hard negative samples, but also leverages the category center of negatives to enhance the robustness of cross-modal representations. • Extensive experiments on two benchmark datasets validate the superiority of our proposed model in comparison to the state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

• This paper introduces the basic concept of graph theory, reviews the properties of Laplacian matrix and the traditional graph cuts method. Starting from four aspects in the realization process of spectral clustering (construction of similarity matrix, establishment of Laplacian matrix, selection of eigenvectors, and determination of the number of clusters), we have summarized in detail some representative algorithms in recent years. • Some successful applications of spectral clustering are summarized. In each aspect, the shortcomings of spectral clustering and some representative improved algorithms are emphatically analyzed. • This paper comprehensively analyzes some research on spectral clustering that has not yet been in-depth, and gives prospects on some valuable research directions. Spectral clustering converts the data clustering problem to the graph cut problem. It is based on graph theory. Due to the reliable theoretical basis and good clustering performance, spectral clustering has been successfully applied in many fields. Although spectral clustering has many advantages, it faces the challenges of high time and space complexity when dealing with large scale complex data. Firstly, this paper introduces the basic concept of graph theory, reviews the properties of Laplacian matrix and the traditional graph cuts method. Then, it focuses on four aspects of the realization process of spectral clustering, including the construction of similarity matrix, the establishment of Laplacian matrix, the selection of eigenvectors and the determination of the number of clusters. In addition, some successful applications of spectral clustering are summarized. In each aspect, the shortcomings of spectral clustering and some representative improved algorithms are emphatically analyzed. Finally, the paper comprehensively analyzes some research on spectral clustering that has not yet been in-depth, and gives prospects on some valuable research directions. [ABSTRACT FROM AUTHOR]

In crowdsourcing scenarios, we can obtain each instance's multiple noisy label set from different workers and then use a ground truth inference algorithm to infer its integrated label. Despite the effectiveness of ground truth inference algorithms, there is still a certain level of noise in integrated labels. To reduce the impact of noise, many noise correction algorithms have been proposed in recent years. To the best of our knowledge, almost all these algorithms assume that workers have the same labeling certainty on different classes and instances. However, it is rarely true in reality due to the differences in workers' individual preferences and cognitive abilities. In this paper, we argue that the labeling certainty of a worker should be class-dependent and instance-dependent. Based on this premise, we propose a certainty weighted voting-based noise correction (CWVNC) algorithm. At first, we use the consistency between worker-labeled labels and integrated labels on different classes to estimate the class-dependent certainty. Then, we train a probability-based classifier on the instances labeled by each worker separately and use it to estimate the instance-dependent certainty. Finally, we correct the integrated label of each instance by weighted voting based on class-dependent certainty and instance-dependent certainty. When the proposed algorithm CWVNC is examined, the average noise ratio of CWVNC on 34 simulated datasets is equal to 15.08%, and on two real-world datasets "Income" and "Music_genre" the noise ratio is equal to 25.77% and 26.94%, respectively. The results show that CWVNC significantly outperforms all other state-of-the-art noise correction algorithms used for comparison. • Crowdsourcing provides an effective way to collect labels from crowd workers. • Noise correction algorithms have been proposed to reduce the noise. • Existing algorithms assume that workers have the same labeling certainty. • This paper proposes a certainty weighted voting-based noise correction algorithm. • The extensive experiments validate the effectiveness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

• A novel learning-based method is proposed for ghost-free high dynamic range (HDR) light field imaging. • A multi-scale architecture integrating deformable alignment module and angular embedding module is designed. • A new large-scale benchmark dataset is established to serve the HDR light field imaging task for dynamic scenes. • The proposed method achieves superior spatial quality and preserves accurate angular consistency. Light field (LF) imaging is an effective way to enable immersive applications. However, limited by the potential well capacity of the image sensor, the acquired LF images suffer from low dynamic range and are thus prone to under-exposure or over-exposure. High dynamic range (HDR) LF imaging is an efficacious avenue to improve the LF imaging's dynamic range. Unfortunately, for dynamic scenes, existing methods are inclined to produce ghosting artifacts and lose details in the saturated regions, while potentially damaging the parallax structure of generated HDR LF images. To address the above challenges, in this paper, we propose a new ghost-free HDR LF imaging method based on a deformable aggregation and angular embedding network. Specifically, considering the four-dimensional geometric structure of the LF image, a deformable alignment module is first designed to handle dynamic regions in the spatial domain, and then the aligned spatial features are fully fused through an aggregation operation. Subsequently, an angular embedding module is constructed to explore angular information to enhance the aggregated spatial features. Based on this, the above two modules are cascaded in a multi-scale manner to achieve multi-level feature extraction and enhance the feature representation ability. Finally, a decoder is leveraged to recover the ghost-free HDR LF image from the enhanced multi-scale features. For performance evaluation, this paper establishes a large-scale benchmark dataset with multi-exposure inputs and ground truth images. Extensive experimental results show that the proposed method generates visually pleasing HDR LF images while preserving accurate angular consistency. Moreover, the proposed method surpasses the state-of-the-art methods in both quantitative and qualitative comparisons. The code and dataset will be available at https://github.com/YeyaoChen/HDRLFI. [ABSTRACT FROM AUTHOR]

• The proposed method combined the heterogeneous data to fully utilizes the texture information of RGB image and the geometric information of point cloud. Compared with depth image, the point cloud has more powerful topology feature, which can be learned with texture feature for accurately and robustly head pose estimation. • The proposed framework is constructed to achieve the feature fusion in the texture-topology level and generate the feature competition among the local regions. This fusion-competition framework enhances the expression of the features with different categories in the different levels to decrease the estimation error and increase the stability. • This paper constructed an RGB-Depth dataset using HoloLens2 for training and testing in head pose estimation. This dataset has abundant head pose samples including 24 sessions with 12 K frames from 21 males and 1 female, and the ground truth of pose in each frame is labeled by an accurate tracking device tied on the head. RGB image and point cloud involve texture and geometric structure, which are widely used for head pose estimation. However, images lack of spatial information, and the quality of point cloud is easily affected by sensor noise. In this paper, a novel fusion-competition framework (FCF) is proposed to overcome the limitations of a single modality. The global texture information is extracted from image and the local topology information is extracted from point cloud to project heterogeneous data into a common feature subspace. The projected texture feature weighted by the channel attention mechanism is embedded into each local point cloud region with different topological features for fusion. The scoring mechanism creates competition among the regions involving local-global fused features to predict final pose with the highest score. According to the evaluation results on the public and our constructed datasets, the FCF improves the estimation accuracy and stability by an average of 13.6 % and 12.7 %, which is compared to nine state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Cell localization in medical image analysis is a challenging task due to the significant variation in cell shape, size and color. Existing localization methods continue to tackle these challenges separately, frequently facing complications where these difficulties intersect and adversely impact model performance. In this paper, these challenges are first reframed as issues of feature misalignment between cell images and location maps, which are then collectively addressed. Specifically, we propose a feature alignment model based on a multi-scale hypergraph attention network. The model considers local regions in the feature map as nodes and utilizes a learnable similarity metric to construct hypergraphs at various scales. We then utilize a hypergraph convolutional network to aggregate the features associated with the nodes and achieve feature alignment between the cell images and location maps. Furthermore, we introduce a stepwise adaptive fusion module to fuse features at different levels effectively and adaptively. The comprehensive experimental results demonstrate the effectiveness of our proposed multi-scale hypergraph attention module in addressing the issue of feature misalignment, and our model achieves state-of-the-art performance across various cell localization datasets. • This paper innovatively addresses the challenges stemming from significant variations in cell shape, scale, and color by reframing them as a feature misalignment problem between cell images and location maps, thereby presenting a unified solution to these complexities. • We propose an innovative multi-scale hypergraph attention module that achieves feature alignment through the adaptive aggregation of features across various scale ranges. • The proposed model achieves state-of-the-art performance on multiple cell localization datasets and reveals great potential. [ABSTRACT FROM AUTHOR]

The low-rank representation (LRR) method has attracted widespread attention due to its excellent performance in pattern recognition and machine learning. LRR-based variants have been proposed to solve the three existing problems in LRR: (1) the projection matrix is permanently fixed when dimensionality reduction techniques are adopted; (2) LRR fails to capture the local geometric structure; and (3) the solution deviates from the real low-rank solution. To address these problems, this paper proposes a low-rank representation with projection distance regularization (PDRLRR) via manifold optimization for clustering. In detail, we first introduce a low-dimensional projection matrix and a projection distance regularization term to fit the projected data automatically and capture the local structure of the data, respectively. Consequently, the projection matrix and representation matrix are obtained jointly. Then, we obtain a more accurate low-rank solution by minimizing the Schatten- p norm instead of the nuclear norm. Next, the projection matrix is optimized through a generalized Stiefel manifold. Extensive experiments demonstrate that our proposed method outperforms the state-of-the-art methods. • This paper proposes a novel PDRLRR model that can simultaneously address the three common problems in LRR. • Data dimensionality reduction technology is integrated into LRR, reducing the data dimensions while learning representation matrices. • For extracting the complete information, the projection distance regularization term is introduced to capture the global and local structure of the data. • The Schatten- p norm instead of the nuclear norm is employed to solve the rank minimization problem of the representation matrix, which can more accurately approximate the real low-rank solution. [ABSTRACT FROM AUTHOR]

Neural architecture search (NAS) has shown excellent performance. However, existing semantic segmentation models rely heavily on pre-training on Image-Net or COCO and mainly focus on the designing of decoders. Directly training the encoder–decoder architecture search models from scratch to SOTA for semantic segmentation requires even thousands GPU days, which greatly limits the application of NAS. To address this issue, we propose a novel neural architecture Search framework for Enhanced Decoder (SED). Utilizing the pre-trained hand-designing backbone and the searching space composed of light-weight cells, SED searches for a decoder which can perform high-quality segmentation. Furthermore, we attach switchable skip connection operations to search space, expanding the diversity of possible network structure. The parameters of backbone and operations selected in searching phrase are copied to retraining process. As a result, searching, pruning and retraining can be done in just 1 day. The experimental results show that the SED proposed in this paper only needs 1/4 of the parameters and calculation in contrast to hand-designing decoder, and obtains higher segmentation accuracy on Cityscapes. Transferring the same decoder architecture to other datasets, such as: Pascal VOC 2012, Camvid, ADE20K proves the robustness of SED. • For the task of image semantic segmentation, we propose a gradient-based, pre-trainable neural network architecture search framework SED. In this paper we simultaneously considering decoder and skip connection search. Our method maximizes the advantages of NAS and pre- trained backbone. • SED can compress the retraining iterations to several thousands. The whole searching, pruning, retraining process can be compressed into 1 day. Furthermore, after searching on Cityscapes, the searched network architecture can achieve 80.2% mIoU. [ABSTRACT FROM AUTHOR]

The guided image filter (GIF) and weighted guided image filter (WGIF) are local linear model-based good edge-preserving filters. However, due to fixed regularization parameter, they suffer from halo artifacts (morphological artifacts) in the sharp regions. To overcome this issue, a robust multi-scale weighting-based edge-smoothing filter (RMWEF) for single image dehazing is proposed in this paper. It removes morphological artifacts and over-smoothness strongly and preserves edge information precisely in both flat and sharp regions. The proposed dehaze method has four-steps. First, initial transmission map and atmospheric map are estimated by using a novel dark channel prior (DCP) method. Then, the morphological artifacts of initial transmission map are reduced by using non-local haze line averaging (NL-HLA) method. In the third step, transmission map is refined by using the proposed RMWEF. Finally, the haze free image is restored. Theoretical and experimental analysis proves that the proposed algorithm produce effective dehaze results quicker than the existing methods. • Robust multi-scale weighting-based edge-smoothing filter (RMWEF) is proposed in this paper. • The value of the cost function (a x ′ , y ′ ) must vary in the range of '0' to '1' depending on the edge- aware smoothing parameter (γ x ′ , y ′ ). The mathematical formulation presented in this paper shows that the proposed filter maintains this relationship, as expected. • This article presents quantitative analysis for three different values of the regularization (ϵ) parameter viz. 0.01 2 , 0.1 2 , 1 2. Thus, it shows the trade-off between regularization parameter (ϵ) and morphological artifacts. • The proposed RMWEF removes morphological artifacts and over-smoothing effects strongly in the fine structures and preserves details in such areas very well by choosing large window radius ζ 1 =60. • The proposed method is tested on 6,618 images from different datasets and the results are compared with 9 existing methods. The experimental results show that it is independent of the nature of images. [ABSTRACT FROM AUTHOR]

• Improving the quality of Federal Learning encrypted alignment data. • Use Gaussian mixture clustering to cluster samples and set a threshold to filter samples. • Use the encrypted sample attribute searching algorithm to fill in the missing value of the sample. • Design the combination model of variation auto-encoder Gaussian hybrid clustering and federated learning. Federated learning can achieve multi-party data-collaborative applications while safeguarding personal privacy. However, the process often leads to a decline in the quality of sample data due to a substantial amount of missing encrypted aligned data, and there is a lack of research on how to improve the model learning effect by increasing the number of samples of encrypted aligned data in federated learning. Therefore, this paper integrates the functional characteristics of deep learning models and proposes a Variational AutoEncoder Gaussian Mixture Model Clustering Vertical Federated Learning Model (VAEGMMC-VFL), which leverages the feature extraction capability of the autoencoder and the clustering and pattern discovery capabilities of Gaussian mixture clustering on diverse datasets to further explore a large number of potentially usable samples. Firstly, the Variational AutoEncoder is used to achieve dimensionality reduction and sample feature reconstruction of high-dimensional data samples. Subsequently, Gaussian mixture clustering is further employed to partition the dataset into multiple potential Gaussian-distributed clusters and filter the sample data using thresholding. Additionally, the paper introduces a labeled sample attribute value finding algorithm to fill in attribute values for encrypted unaligned samples that meet the requirements, allowing for the full recovery of encrypted unaligned data. In the experimental section, the paper selects four sets of datasets from different industries and compares the proposed method with three federated learning clustering methods in terms of clustering loss, reconstruction loss, and other metrics. Tests on precision, accuracy, recall, ROC curve, and F1-score indicate that the proposed method outperforms similar approaches. [ABSTRACT FROM AUTHOR]

Facial expression recognition (FER) is one of the most important computer vision tasks for understanding human inner emotions. However, the poor generation ability of the FER model limits its applicability due to tremendous intraclass variation. Especially for expressions of varying intensities, the appearance differences among weak expressions are subtle, which makes FER tasks challenging. In response to these issues, this paper presents a dual subspace manifold learning method based on a graph convolutional network (GCN) for intensity-invariant FER tasks. Our method treats the target task as a node classification problem and learns the manifold representation using two subspace analysis methods: locality preserving projection (LPP) and peak-piloted locality preserving projection (PLPP). Inspired by the classic LPP, which maintains local similarity among data, this paper introduces a novel PLPP that maintains the locality between peak expressions and non-peak expressions to enhance the representation of weak expressions. This paper also reports two subspace fusion methods, one based on a weighted adjacency matrix and another on a self-attention mechanism, that combine the LPP and PLPP to further improve FER performance. The second method achieves a recognition accuracy of 93.83% on the CK+, 74.86% on the Oulu-CASIA and 75.37% on the MMI for weak expressions, outperforming state-of-the-art methods. • A semi-supervised learning framework based on a GCN for intensity-invariant FER tasks. • A novel PLPP method to keep the locality between peak and non-peak expressions. • Two different subspace fusion methods that combine the LPP and PLPP results. [ABSTRACT FROM AUTHOR]

In many high-dimensional datasets, the phenomenon that features are relevant often occurs. Elastic-net regularization is widely used in support vector machines (SVMs) because it can automatically perform feature selection and encourage highly correlated features to be selected or removed together. Recently, some effective algorithms have been proposed to solve the elastic-net SVMs with different convex loss functions, such as hinge, squared hinge, huberized hinge, pinball and huberized pinball. In this paper, we develop a linearized alternating direction method of multipliers (LADMM) algorithm to solve above elastic-net SVMs. In addition, our algorithm can be applied to solve some new elastic-net SVMs such as elastic-net least squares SVM. Compared with some existing algorithms, our algorithm has comparable or better performances in terms of computational cost and accuracy. Under mild conditions, we prove the convergence and derive convergence rate of our algorithm. Furthermore, numerical experiments on synthetic and real datasets demonstrate the feasibility and validity of the proposed algorithm. • We develop a linearized ADMM algorithm for effectively solving a series of elastic-net SVMs. • The proposed algorithm is accompanied by convergence and complexity analysis. • A new elastic-net SVM called LESVM is proposed in this paper. [ABSTRACT FROM AUTHOR]

In pattern recognition tasks, the information from system input is modeled through a series of nonlinear operations, which include but not limited to feature extraction, regression, and classification. Both theoretically and practically, these operations are inevitably subject to internal modeling error and external disturbance, resulting at a performance challenge. Those state-of-the-art methods, e.g. Convolutional Neural Network and Transformer, still display significant instabilities and failures under practical applications, so comes a lack of generalization. Consequently, the more robust pattern recognition methods and related theories still merit a further study. This paper firstly reviews those state-of-the-art technologies in the field. The bottleneck of performances in those latest researches is associated with a lack of disturbance estimation and corresponding compensation. Therefore, the implications of disturbance rejection in pattern recognition field are further discussed from a control point of view. Then, the open problems are summarized. Ultimately, a discussion of the potential solutions, which is related to the application of compensation on features, is given to highlight the future study. Through the systematic review in this paper, the disturbance rejection in pattern recognition is developed into a control problem. Hopefully, more effective control technologies for the compensation on features can be used to improve the robustness of pattern recognition theoretically and practically. • The pattern recognition process is inevitably subject to internal modeling error and external disturbance. • The shortcomings of existing disturbance rejection technologies are summarized. • The bottleneck of performances in those latest researches is associated with a lack of disturbance estimation and corresponding compensation. • The open problems and the potential solutions are summarized to discuss the future study in an innovative control point of view, i.e. a compensation on features. [ABSTRACT FROM AUTHOR]

Irregularly Sampled Time Series (ISTS) include partially observed feature vectors caused by the lack of temporal alignment across dimensions and the presence of variable time intervals. Especially in medical applications, because patients' examinations depend on their health status, observations in this event-based medical time series are nonuniformly distributed. When using deep learning models to classify ISTS, most work defines the problem that needs to be solved as alignment-caused data missing or nonuniformity-caused dependency change. However, they only modeled relationships between observed values, ignoring the fact that time is the independent variable for a time series. In this paper, we emphasize that irregularity is active, time-depended, and class-associated and is reflected in the Time Pattern (TP). To this end, this paper focused on the TP of ISTS for the first time, proposing a Time Pattern Reconstruction (TPR) method. It first encodes time information by the time encoding mechanism, then imputes values from time codes by the continuous-discrete Kalman network, selects key time points by the conditional masking mechanism, and finally classifies ISTS based on the reconstructed TP. Experiments on four real-world medical datasets and three other datasets show that TPR outperforms all baselines. We also show that TP can reveal biomarkers and key time points for diseases. • Classifying irregularly sampled time series is crucial in real-world applications. • Incorporate time-depended mapping into time series classification. • Extract time patterns of irregularly sampled time series. • Propose a plug-in for time-series deep learning models. • Promote the knowledge discovery ability of medical diagnosis methods. [ABSTRACT FROM AUTHOR]

Deep convolutional neural networks (CNNs) have been widely applied to cataract recognition tasks and have achieved promising results. However, most existing methods focused on designing data-driven CNN architectures, and failed to exploit asymmetric opacity distribution prior of cataract, which is significant for cataract diagnosis. To this end, this paper proposes a regional context-based recalibration (RCR) module, which fully leverages the clinical prior to recalibrate the feature maps with regional pooling, region-based context integration, and integrated context fusion. We stack these RCR modules to form an RCRNet based on anterior segment optical coherence tomography (AS-OCT) images for cataract recognition. Experiments on the AS-OCT-NC2 dataset and two publicly available medical datasets demonstrate that RCRNet achieves a better trade-off between performance and efficiency than state-of-the-art channel attention-based networks. We also explain the inherent behavior of RCRNet with the aid of the visual analysis. In addition, this paper is the first to study the effects of two performance evaluation methods on AS-OCT image-based cataract classification results: the single-image level and the single-eye level, suggesting that adopting the single-eye level to evaluate cataract classification performance according to clinical diagnosis requirement. • We design a regional context-based recalibration module by leveraging the prior of cataract. • We construct an RCRNet for cataract recognition based on AS-OCT images. • We test the effects of two performance evaluation methods on cataract classification performance. [ABSTRACT FROM AUTHOR]

Deep clustering has shown its promising capability in joint representation learning and clustering via deep neural networks. Despite the significant progress, the existing deep clustering works mostly utilize some distribution-based clustering loss, lacking the ability to unify representation learning and multi-scale structure learning. To address this, this paper presents a new deep clustering approach termed I mage c luster i ng with c ontrastive le arning and multi-scale G raph C onvolutional N etworks (IcicleGCN), which bridges the gap between convolutional neural network (CNN) and graph convolutional network (GCN) as well as the gap between contrastive learning and multi-scale structure learning for the deep clustering task. Our framework consists of four main modules, namely, the CNN-based backbone, the Instance Similarity Module (ISM), the Joint Cluster Structure Learning and Instance reconstruction Module (JC-SLIM), and the Multi-scale GCN module (M-GCN). Specifically, the backbone network with two weight-sharing views is utilized to learn the representations for the two augmented samples (from each image). The learned representations are then fed to ISM and JC-SLIM for joint instance-level and cluster-level contrastive learning, respectively, during which an auto-encoder in JC-SLIM is also pretrained to serve as a bridge to the M-GCN module. Further, to enforce multi-scale neighborhood structure learning, two streams of GCNs and the auto-encoder are simultaneously trained via (i) the layer-wise interaction with representation fusion and (ii) the joint self-adaptive learning. Experiments on multiple image datasets demonstrate the superior clustering performance of IcicleGCN over the state-of-the-art. The code is available at https://github.com/xuyuankun631/IcicleGCN. • This paper for the first time enables multi-scale structure learning for image clustering via GCNs. • Multi-scale structure learning and two levels of contrastive learning are jointly enforced. • A novel deep image clustering approach termed IcicleGCN is proposed. • Extensive experimental results confirm the superiority of IcicleGCN over the state-of-the-art. [ABSTRACT FROM AUTHOR]

• We would like to submit the manuscript entitled "High-order relational generative adversarial network for video super-resolution", which we wish to be considered as a research article for publication in "Pattern Recognition". Because our contributed research covers the area" Image processing and representation", this manuscript belongs in this journal. No conflict of interest exits in the submission of this manuscript, and manuscript is approved by all authors for publication. • Both the instability of motion compensation and feature aggregation often undermine the SR results of videos. Only the global spatio-temporal dependencies of same scale features are considered as one-order relations, with difficulty in handling the complex motions and context variations. To deal with these issues, we argue that high-order relations should be modelled and applied in the generative strategies to exploit feature alignments and long-range information fusion. In fact, the high-order relations mean that the dependencies of features among pixel positions are mined not only from global feature representations but also through local patch interactions. Moreover, the stronger relations can be further revealed across different scales, by which the aggregation weight matrices with high-order statistics are adaptively determined. The videos are regarded as the complementary fusion of motion and context. Thus, simultaneously capturing the underlying relations of the motion and context is the key to video SR. In this paper, we propose a high-order relational generative adversarial network (HOR-GAN) for accurate video SR, which has strong capacity of alleviating the erroneous motion compensation and merging the useful information among the consecutive frames. • In summary, we highlight the main contributions of this paper as follows: We propose a HOR-GAN framework for highly accurate and realistic video SR. By exploiting high-order relations of feature patches under the construction of the pyramid graphs, the proposed HOR-GAN works well to produce a better feature alignment and fuse more spatio-temporal information. We adopt the dual discriminators to provide spatially coherent feedback to the generator, thus making the generator focus more on fine-grained features. The effectiveness of the proposed method has been justified through the extensive experiments. • We design a motion-aware relation module to accurately align neighboring frames with the reference ones. In this module, a patch-wise matching strategy is first used to build cross-scale correspondences of similar patches. The graph attention layers then adaptively aggregate the local patch features to further decrease the alignment error. We develop a context-aware relation module to make full use of high-order dependencies among all warped feature patches. We introduce the multi-scale graph convolution layers to mine contextual interaction relations for better fusing the spatio-temporal features, in which the position information is encoded to augment the detail recovery. Finally, each pixel is enhanced via a global self-attention. • We deeply appreciate your consideration of our manuscript, and we look forward to receiving comments from the reviewers. If you have any queries, please don't hesitate to contact me. Yours sincerely, Rui Chen Video super-resolution can reconstruct a sequence of high-resolution frames with temporally consistent contents from their corresponding low-resolution sequences. The key challenge for this task is how to effectively utilize both inter-frame temporal relations and intra-frame spatial relations. The existing methods for super-resolving the videos commonly estimate optical flows to align the features of multiple frames based on temporal correlations. However, motion estimation is often error-prone and hence largely hinders the recovery of plausible details. Moreover, high-order contextual dependencies in the feature space are rarely exploited for further enhancing the spatio-temporal information fusion. To this end, we propose a novel generative adversarial network to super-resolve low-resolution videos, which makes full use of patch embeddings and is effective in exploring high-order spatio-temporal relations of the feature patches. Specifically, a motion-aware relation module is designed to handle the alignment between neighboring frames and reference ones. Depending on a patch-matching strategy for adaptive selection of multiple most similar patches, the cross-scale graph is constructed to reliably aggregate these patches using a feature pyramid. Based on the structure of multi-scale graph, a context-aware relation module is developed to capture high-order dependencies among resulting warped patches for better leveraging long-range complementary contexts. To further enhance reconstruction ability, the temporal position information of video sequences is also encoded into this module. Dual discriminators with cycle consistent constraints are adopted to provide more informative feedback to the generator while maintaining the global coherence. Extensive experiments have demonstrated the effectiveness of the proposed method in terms of quantitative and qualitative evaluation metrics. [ABSTRACT FROM AUTHOR]

Automatic classifier accuracy evaluation (ACAEval) on unlabeled test sets is critical for unseen real-world environments. The use of dataset-level regression on synthesized meta-datasets (comprised of many sample sets) has shown promising results for ACAEval. However, the existing meta-dataset for ACAEval is created using simple image transformations such as rotation and background substitution, which can make it difficult to ensure a reasonable distribution shift between the sample set and the test set. When the distribution shift is large, it becomes challenging to estimate the classifier accuracy on the test set using those sample sets. To ensure more robust ACAEval, this paper attempts to customize a meta-dataset in which each sample set has a reasonable distribution shift to the test set. An intra-class cycle-consistent adversarial learning (ICAL) method is introduced to transfer the style of a labeled training set to the style of the test set, by jointly considering the domain shift issue, the label flipping issue (the semantic information may be changed after style transformation), and the diversity of multiple sample sets in the meta-dataset. Experiments validate that under the same experimental setup, our method outperforms the existing ACAEval methods by a good margin, and achieves state-of-the-art performance on several standard benchmark datasets, including digit classification and natural image classification. • Two authors named Yan Huang in pinyin, one PostDoc (1st author), the other an Associate Prof (3rd author). • Paper on ACAEval for classifier accuracy. Uses meta-dataset technique for unlabeled real-world data. • Customized meta-dataset for ACAEval to address distribution shift between samples and test set. • Our sample set considers label flip issue. Introduces random indicator and FD margin loss. • Experiments demonstrate our ACAEval matches existing methods and surpasses dataset-level regression. [ABSTRACT FROM AUTHOR]

• The paper proposes a hard positives oriented contrastive learning strategy for semi-supervised medical image segmentation. • The HPC strategy is constructed from two levels: an unsupervised image-level HPC and a supervised pixel-level HPC. • The pixel-level HPC is implemented in a region-based manner to save memory and deliver more multi-granularity information. • The paper inserts several feature swap modules into the pre-trained decoder to encourage robust segmentation predictions. • The proposed framework outperforms the state-of-the-art methods on two public clinical datasets. Semi-supervised learning (SSL) has been a popular technique to resolve the annotation scarcity problem in pattern recognition and medical image segmentation, which usually focuses on two critical issues: 1) learning a well-structured categorizable embedding space, and 2) establishing a robust mapping from the embedding space to the pixel space. In this paper, to resolve the first issue, we propose a h ard p ositives oriented c ontrastive (HPC) learning strategy to pre-train an encoder-decoder-based segmentation model. Different from vanilla contrastive learning tending to focus only on hard negatives, our HPC learning strategy additionally concentrates on hard positives (i.e., samples with the same category but dissimilar feature representations to the anchor), which are considered to play an even more crucial role in delivering discriminative knowledge for semi-supervised medical image segmentation. Specifically, the HPC is constructed from two levels, including an unsupervised image-level HPC (IHPC) and a supervised pixel-level HPC (PHPC), empowering the embedding space learned by the encoder with both local and global senses. Particularly, the PHPC learning strategy is implemented in a region-based manner, saving memory usage while delivering more multi-granularity information. In response to the second issue, we insert several feature swap (FS) modules into the pre-trained decoder. These FS modules aim to perturb the mapping from the intermediate embedding space towards the pixel space, trying to encourage more robust segmentation predictions. Experiments on two public clinical datasets demonstrate that our proposed framework surpasses the state-of-the-art methods by a large margin. Source codes are available at https://github.com/PerPerZXY/BHPC. [ABSTRACT FROM AUTHOR]

This paper proposes an edge based compressively sensed (CS) geodesic active contour (GAC) model, termed CS-GAC, to ensure faithful edge detection and accurate object segmentation. The motivation behind this paper is that edge information driving the contour evolution can be iteratively obtained by incomplete CS measurements. In each iteration, the CS-GAC is a three-step process including edge detection, active contouring and sparse reconstruction. Instead of working on the final reconstructed images themselves, the evolution of the CS-GAC is driven by a few CS measurements and guided by updatable edge information. The edge information is generated by a complex shearlet transform (CST) based edge map. In the framework, reconstruction and edge detection work alternately. The iterative update property that takes advantages of both edge sparsity and edge detection can largely improve the evolution precision. Numerical experiments show that the CS-GAC can obtain challenging segmentation results in comparisons with the state of the art methods, and has competitive prospects. • Edge information driving the contour evolution is iteratively obtained from CS measurements. • Updating property of the edge indicator takes advantages of both edge sparsity and edge detection. • The complex shearlet transform based edge map has been improved by the iteratively updating mechanism. [ABSTRACT FROM AUTHOR]

Recently, new sensors with active pixels were brought to market. These sensors export local variations of light intensity in the form of asynchronous events with low latency. Since the data output format is a stream of addressable events and not a complete image of light intensities, new algorithms are required for known problems in the field of Computer Vision, such as segmentation, VO, SLAM, object, and scene recognition. There are some proposed methodologies for object recognition using conventional methods, convolutional neural networks, and third-generation neural networks based on spikes. However, convolutional neural networks and spike neural networks require specific hardware for processing, hard to miniaturize. Also, several traditional Computer Vision operators and feature descriptors have been neglected in the context of event sensors and could contribute to lighter methodologies in object recognition. This paper proposes an algorithm for local binary pattern extraction in sparse structures, typically found in this context. This paper also proposes two methodologies using local binary patterns to captures with event-based sensors for object recognition. The first methodology exploits the known motion performed by the sensor, while the second is motion agnostic. It is demonstrated experimentally that the LBP operator is a fast and light alternative that enables variable reduction using PCA in some cases. The experiments also show that it is possible to reduce the final feature vector for classification by up to 99 , 73 % when compared to conventional methods considered state-of-the-art while maintaining comparable accuracy. • Sparse-LBP extraction algorithm. • Methodologies for memory-efficient object recognition with event-based sensors. • Surface of active events representation using hash tables. [ABSTRACT FROM AUTHOR]

• LPSF module is designed to improve the gas features. • Collaborative attention is proposed for local and global gas features. • The feature compensation mechanism of CNN and Transformer is designed. • PSCFormer is proposed for gas recognition. Based on their powerful feature extraction capability, a convolutional neural network (CNN) has been gradually applied to gas identification in the electronic nose (e-nose) system. The responses of different intensities in the e-nose system are significantly correlated, and CNN extracts the local gas features by convolution while ignoring their global correlation. Transformer combines different responses and obtains the correlation between global features by self-attention. This paper proposes a lightweight hybrid network called Peak Search-based Convolutional Transformers (PSCFormer). First, combining the data characteristics of gas information, the Local Peak Search and Feature Fusion (LPSF) module is proposed to focus on the key gas features. Second, Transformer Encoder (TE) is proposed to obtain the global correlation between global features, and the parallel Convolution Encoder (CE) is proposed to capture the local dependence. Finally, a reasonable feature complementation mechanism is presented, and the preference of TE is alleviated for the slow-down response while solving the receptive field limitation of CE. This paper has evaluated three different datasets to validate the effectiveness of PSCFormer, all of which show stable and excellent performance with a good tradeoff between efficiency and complexity. The results prove that PSCFormer is an efficient and lightweight gas identification network, which provides a method to promote the engineering application of the e-nose system. [ABSTRACT FROM AUTHOR]

How to quickly obtain information about crisis events on social media such as Twitter and Weibo is crucial for follow-up rescue work and the promotion of postdisaster reconstruction. Therefore, it is very important to obtain useful information through multimodal summary generation technology. The current technology for generating crisis event summaries is mainly affected by unimodal bias and disregards the diversity of information in text and images. To solve these problems, this paper proposes a hierarchical multimodal crisis event summary generation model based on the modal alignment premise and hierarchical thinking. First, the visual context vector and text context vector are obtained, and then the hierarchical multimodal pointer model is employed to generate the text summary. Thus, the modal deviation is solved. Second, to select high-quality images, this paper proposes a dynamic selection strategy, which to some extent considers the requirements of the high correlation between text and images and the diversity of crisis information. Last, the experimental results based on the crisis event data in the MSMO dataset show that the proposed model achieves good performance in the summary generation and image selection of crisis events. • This paper proposes a multimodal hierarchical fusion model for crisis event summarization, which maintains the independence of images. • This paper proposes a dynamic selection strategy to better select suitable images. [ABSTRACT FROM AUTHOR]

• We propose a human identification method based on dental impression images. First, we extract a geometric feature called the calibrated offset distance to describe the dental arch. The dental arch is a distinctive feature of individual teeth. To address the challenges in describing the dental arch, this paper introduces a novel calibrated offset distance that captures the characteristics of the arc, changes in radian, and distances between arc breakpoints simultaneously. • We propose an adaptive threshold strategy for the tooth print detection model based on the YOLOv8 network. We design a gradient training strategy to achieve a robust detection model. Building upon its high positioning accuracy, we propose an adaptive threshold strategy to further enhance the detection accuracy. • We propose a multi-scale and multi-dimensional tooth print aggregation feature. In order to enhance the ability to recognize individuals with similar teeth structures, we further integrate SIFT features to provide a detailed description based on geometric features. This aggregation feature is composed to improve the performance of human identification. In large-scale natural disasters and special criminal cases, surface features of bodies, such as faces and fingerprints, are easily destroyed. Teeth possess strong high-temperature resistance, corrosion resistance, and high hardness, which can compensate for the shortcomings of the aforementioned situations. This paper proposes an identification method based on the aggregated features of multi-scale dental impression images. Firstly, a method exploiting the adaptive object detection method based on YOLOv8 is proposed to segment toothprints. Next, a novel geometric feature named calibrated offset distance is extracted, combined with the SIFT feature method, to extract multi-scale and multi-dimensional features from the global toothprint, local toothprints, and single-tooth prints. Finally, all features are aggregated to enhance the descriptive ability and robustness. Experimental results indicate that the method proposed in this paper demonstrates good identification performance. [ABSTRACT FROM AUTHOR]

• This paper explores the potential of spatial feature dimensions to seek an effective feature representation for point clouds. • The dimension expansion of MKConv enables spatially correlated feature representation and discrete convolutions for point cloud data without information loss. • For stable and effective learning, kernel weight normalization and multidimensional attention are proposed. • The effectiveness of multidimensional feature representation is demonstrated by the superior performance on three different point cloud processing tasks including object classification, object part segmentation, and scene semantic segmentation. Despite the remarkable success of deep learning, an optimal convolution operation on point clouds remains elusive owing to their irregular data structure. Existing methods mainly focus on designing an effective continuous kernel function that can handle an arbitrary point in continuous space. Various approaches exhibiting high performance have been proposed, but we observe that the standard pointwise feature is represented by 1D channels and can become more informative when its representation involves additional spatial feature dimensions. In this paper, we present Multidimensional Kernel Convolution (MKConv), a novel convolution operator that learns to transform the point feature representation from a vector to a multidimensional matrix. Unlike standard point convolution, MKConv proceeds via two steps. (i) It first activates the spatial dimensions of local feature representation by exploiting multidimensional kernel weights. These spatially expanded features can represent their embedded information through spatial correlation as well as channel correlation in feature space, carrying more detailed local structure information. (ii) Then, discrete convolutions are applied to the multidimensional features which can be regarded as a grid-structured matrix. In this way, we can utilize the discrete convolutions for point cloud data without voxelization that suffers from information loss. Furthermore, we propose a spatial attention module, Multidimensional Local Attention (MLA), to provide comprehensive structure awareness within the local point set by reweighting the spatial feature dimensions. We demonstrate that MKConv has excellent applicability to point cloud processing tasks including object classification, object part segmentation, and scene semantic segmentation with superior results. [ABSTRACT FROM AUTHOR]

• We propose an interpolation pyramid algorithm to capture multi-scale information from DSMs. • Multi-scale information obtained from DSMs is leveraged to guide normal segmentation network. • We use attention mechanism to reduce misleading guidance information caused by noise in DSMs. • Our method attains state-of-the-art performance on the Potsdam and Vaihingen datasets. Compared with other kinds of images, aerial images have more obvious object scale distinction and larger resolution, which results in that the whole scale information of aerial images can hardly be explored. To address this difficulty, we develop a novel network based on the digital surface models (DSMs) of aerial images in this paper. The proposed network termed MSINet can efficiently mine scale information through the DSMs from two aspects. Firstly, we propose an interpolation pyramid algorithm to encode the scale information from the DSMs and hence provide a scale prior information to the normal segmentation network. The interpolation pyramid algorithm implements interpolation operations with different scales on the DSMs and detects the pixel value change after the interpolation operations. Objects with different scales will express diverse changes, which provides useful information to encode their scale information. Secondly, aiming to address the problem that the DSMs contain a large amount of noise in the boundary part, a spatial information enhancement module and a mutual-guidance module are developed in this paper. These two modules can fix the misleading guidance information caused by the noise in the boundary part of the DSMs and hence achieve more accurate scale information inserting. The extensive experimental results prove that our methods can outperform other competitors in terms of qualitative and quantitative performance. [ABSTRACT FROM AUTHOR]

• This paper focuses on fine-grained visual recognition by learning from noisy and long-tail distributed training sets. It proposes the multi-proxy feature learning (MPFL) strategy, which learns multiple proxy features instead of a uniform representation for each category to eliminate noisy labels and pursue a more balanced and discriminative feature space. • The MPFL includes two modules compact memory updater (CMU) and proxy-based feature enhancement (PFE). The CMU maintains a memory bank, which memorizes proxy features to represent multiple typical appearances of each category in the training set. The PFE hence leverages proxy features to optimize intra and inter category distances, and ensures samples of the same category have more similar features. • Extensive experiments on fine-grained visual recognition tasks including person re-identification (re-id), image classification and retrieval show that the proposed method gets remarkable improvements when training with noisy labels and long-tailed distributions. Visual representation for fine-grained visual recognition can be learned by mandatorily enforcing all samples of the same category into a uniform representation. This strict training objective performs well under closed-set setting but is not applicable to data in the wild containing noisy annotations and long-tailed distributions, e.g., it may lead to a feature space biased to head categories. This paper tackles this challenge by pursuing a more balanced and discriminative feature space by first retaining intra-class variances to isolate noises, then eliminating intra-class variances to improve the visual recognition performance. We propose the Compact Memory Updater to maintain a memory bank, which memorizes proxy features to represent multiple typical appearances of each category in the training set. The Proxy-based Feature Enhancement hence leverages proxy features to ensure samples of the same category have similar features. Iteratively running those two modules boosts the robustness and discriminative power of the learnt representation, hence facilitates various fine-grained visual recognition tasks including person re-identification (re-id), image classification and retrieval. Extensive experiments on noisy and long-tailed training sets show this Multi-Proxy Feature Learning (MPFL) framework achieves promising performance. For instance on a training set with 90% one-shot categories, MPFL outperforms the recent long-tailed person re-id method LEAP-AF by 16.9% in rank-1 accuracy. [ABSTRACT FROM AUTHOR]

• Space-time discrete switching complex networks is newly modeled. • Clusters synchronous frame to discrete spatial models is built in the first time. • Boundary feedback controllers with switching and clustering signals are designed. • Networks coupled with both nodes' states and spatial diffusions are explored. • A computer algorithm is designed to implement the addressed results of this paper. Unlike the existing literatures that consider only discrete-time networks, this paper explores the double effects of both discrete time and discrete spatial diffusions in a switching complex dynamical networks. By means of the knowledge of clusters controls, a clusters synchronous frame of space-time discrete switching complex networks with boundary feedback controller is newly proposed and established. With the helps of some indispensable vector-valued sequence inequalities and Lyapunov function with switching signals and clusters' information, the boundary feedback controllers are designed to synchronize space-time discrete switching complex networks coupled with nodes' states or spatial diffusions in the form of clusters. Additionally, a realizable computer algorithm is given to make the derived results of this paper easier to enforce. The current work is pioneering in consideration of discrete spatial diffusions and provides a theoretical and practical basis for future research in this regard. [ABSTRACT FROM AUTHOR]

• The paper provides fuzzy c-means algorithms based on Gaussian kernel functions. • The first algorithm computes the width parameters though suitable constraints. • The second algorithm computes the width parameters though entropy regularization. • Experiments with benchmark data sets shows the usefulness of the algorithms. The conventional Gaussian kernel fuzzy c-means clustering algorithms require selecting the width hyper-parameter, which is data-dependent and fixed for the entire execution. Not only that, but these parameters are the same for every dataset variable. Therefore, the variables have the same importance in the clustering task, including irrelevant variables. This paper proposes a Gaussian kernel fuzzy c-means with kernelization of the metric and automated computation of width parameters. These width parameters change at each iteration of the algorithm and vary from each variable and from each cluster. Thus, this algorithm can re-scale the variables differently, thus highlighting those that are relevant to the clustering task. Fuzzy clustering algorithms with regularization have become popular due to their high performance in large-scale data clustering, robustness for initialization, and low computational complexity. Because the width parameters of the variables can also be controlled by entropy, this paper also proposes Gaussian kernel fuzzy c-means algorithms with kernelization of the metric and automated computation of width parameters through entropy regularization. To demonstrate their usefulness, the proposed algorithms are compared with the conventional KFCM-K algorithm and previous algorithms that automatically compute the width parameter of the Gaussian kernel. [ABSTRACT FROM AUTHOR]

• The feature – Edge, which can represent the whole point cloud is introduced into the iterative calculation to reduce the mismatch between corresponding points and improve the convergence speed. • The variance minimization objective function based on bi-direction of normal and tangential distance is used to make the allowance distribution uniform while reducing the tangential runout of the point cloud. • In order to make up for missing points when extracting the edge of point cloud, the extracted edge points are converted into curvature space. Curvature octree is used to search for points with similar curvature degrees, so that the set of points sufficient to represent the global point cloud can participate in the subsequent calculation. The basis for guidance in the field of automated robot processing is modeling by visual scanning. Matching algorithms are the real link that can be made between the ideal model and the subsequent robot processing. The matching algorithm that meets the machining requirements plays a pivotal role in the entire process, which provides the exact location of design models and measurement data. In order to meet the requirement of making the machining allowance uniform, a fine registration method which considers the variance minimization of the normal and tangential distances between the edge neighbors of the scattered point cloud is proposed. The edge neighbors are achieved by local growing of edge seeds based on the minimization of energy of supervoxel, which can represent the model more accurate than edge points extracted directly. The edge neighbor points are used to participate in the calculation, and the objective function of minimizing the variance of the two-way distance with the introduction of weight coefficients is proposed to constrain the iterative process. The effect of the distance in two directions on the result is analyzed, to determine the appropriate weight coefficients so that the matching calculation converges quickly and accurately. In comparison with other classical and state-of-the-art matching methods, the method in this paper performs well in terms of solution efficiency and accuracy of results. Moreover, the ability of this paper's method to resist Gaussian noise is investigated, and it is found that this paper's method has good robustness when σ is less than 1 for Gaussian noise. Ultimately, a uniformly distributed residual model is obtained to provide a visually guided basis for subsequent machining. [ABSTRACT FROM AUTHOR]

• This paper opens a new door for studying PageRank based on the spectral distribution law of random matrices. • We provide a method for pairwise comparisons in PageRank in O (1) without computing the exact value of the principle eigenvectors of Google matrices from a probabilistic view. • We provide a method for extracting the top k list in O (kn), where n is the total number of involved items This paper shows that pairwise PageRank orders emerge from two-hop walks. The main tool used here refers to a specially designed sign-mirror function and a parameter curve, whose low-order derivative information implies pairwise PageRank orders with high probability. We study the pairwise correct rate by placing the Google matrix G in a probabilistic framework, where G may be equipped with different random ensembles for model-generated or real-world networks with sparse, small-world, scale-free features, the proof of which is mixed by mathematical and numerical evidence. We believe that the underlying spectral distribution of aforementioned networks is responsible for the high pairwise correct rate. Moreover, the perspective of this paper naturally leads to an O (1) algorithm for any single pairwise PageRank comparison if assuming both A = G − I n , where I n denotes the identity matrix of order n , and A 2 are ready on hand (e.g., constructed offline in an incremental manner), based on which it is easy to extract the top k list in O (kn), thus making it possible for PageRank algorithm to deal with super large-scale datasets in real time. [ABSTRACT FROM AUTHOR]

• We have proposed a Hard and Easy Negative samples mining based Distance learning (HEND) approach for person re-identification. • We have designed a symmetric triplet constraint for the proposed HEND approach. • We have proposed a Projection based HEND (PHEND) approach, which simultaneously learns a projection matrix and a distance metric. • We have conducted extensive experiments in this paper to evaluate our approaches. Distance learning is an effective technique for person re-identification. In practice, the hard negative samples usually contain more discriminative information than the easy negative samples. Therefore, it's necessary to investigate how to make full use of the discriminative information conveyed by different types of negative samples in the distance learning process. In this paper, we propose a H ard and E asy N egative samples mining based D istance learning (HEND) approach for person re-identification, which learns the distance metric by designing different objective functions for hard and easy negative samples, such that the discriminative information contained in negative samples can be exploited more effectively. Moreover, considering that there usually exist large differences between the images captured by different cameras, we further propose a projection-based HEND approach to reduce the influence of between-camera differences to the re-identification. Experimental results on seven pedestrian image datasets demonstrate the effectiveness of the proposed approaches. [ABSTRACT FROM AUTHOR]

• This paper reports the first attempt in integrating multiobjective opti- mization (MOO) and genomic structure for solving the subspace clustering problem. • Multiobjective optimization framework is utilized to simultaneously opti- mize several sub-space cluster quality measures. • The existing cluster quality measures are modified to handle subspace clustering problem. • As a part of the experiments, the proposed algorithm is applied on the bi-clustering of gene expression data to show the efficacy of the existing technique in solving some real-life problem. Bi-clustering of gene expres- sion data is a sub-space clustering problem where a subset of rows and a subset of columns need to be selected. Subspace clustering techniques become paramount in pattern recognition for detecting local variations from high dimensional data. Several techniques exist in the recent literature for subspace clustering, majority of which optimize implicitly or explicitly a single cluster quality measure. Inspired by the success of multi-objective optimization in solving clustering problem, we developed a multi-objective based subspace clustering technique in this paper. The proposed technique simultaneously optimizes two subspace cluster quality measures, capable of capturing different cluster shapes/properties. Two existing cluster quality measures, XB-index and PBM-index, are modified to develop subspace cluster validity indices, and then those are used as optimization criteria. These cluster validity indices measure the appropriateness of generated subspace clusters in terms of intra-subspace cluster similarity and separation between subspace clusters. The proposed approach utilizes a new evolvable genome structure which stores the information about subspaces in its phenotype and genotype and evolves this genome structure with the help of different genetic operators. The developed algorithm is applied on ten standard real-life data sets and sixteen synthetic datasets for identifying different subspace clusters. The results obtained by this algorithm are compared against some state-of-the-art techniques with respect to different performance metrics. Experimentation reveals that the proposed algorithm is able to take advantages of its evolvable genomic structure and multi-objective based framework and it can be applied to any data set. In a part of the paper, the efficacy of the proposed technique is also shown for bi-clustering of gene-expression data sets. [ABSTRACT FROM AUTHOR]

• A method is introduced to hallucinate super-low face from real low-quality face instead of stimulated low-quality face. • A definition-scalable strategy: a face is decomposed into a basic face with low-definition and an enhanced face with high-frequency information. • The super-resolution technique based on definition-scalable inference effectively estimate structural information and high-frequency texture from real low-res faces. • The matched SIFT key-points is proposed to estimate the similarity of the super-res face and its high-res labeled face. • The proposed SISR method can recover more structure information and local information from real low-quality face and more SIFT key-points than the state of the arts. To hallucinate super-resolution (super-res) face from a real low-quality face, a super-resolution technique based on definition-scalable inference (SRDSI) is proposed in this paper. In the proposed strategy, all high-res labeled faces are first decomposed into basic faces and enhanced faces to train a basic face and an enhanced face inferring model, and then two inferring models are used to hallucinate super-res basic face with low-definition and enhanced faces with high-frequency information from a single low-res face. Finally, the basic face is merged with its enhanced face into a super-res face with high-definition. In addition, this paper employs SIFT key-points to evaluate the similarity between the super-res face and its high-res labeled face. Experimental results show that SRDSI can effectively recover more structural information as well as SIFT key-points from real low-res faces and achieves better performance than state-of-the-art super-resolution techniques in terms of both visual and objective quality. Image, graphical abstract [ABSTRACT FROM AUTHOR]

• Handwritten Text Recognition is researched in this paper with a set of free available benchmarks. • Freely available tools are provided for Handwritten Text Recognition. • Competitive results are provided with Convolutional Recurrent Neural Networks and N-gram language models. • New challenges are described related to Handwritten Text Recognition. Handwritten Text Recognition is a important requirement in order to make visible the contents of the myriads of historical documents residing in public and private archives and libraries world wide. Automatic Handwritten Text Recognition (HTR) is a challenging problem that requires a careful combination of several advanced Pattern Recognition techniques, including but not limited to Image Processing, Document Image Analysis, Feature Extraction, Neural Network approaches and Language Modeling. The progress of this kind of systems is strongly bound by the availability of adequate benchmarking datasets, software tools and reproducible results achieved using the corresponding tools and datasets. Based on English and German historical documents proposed in recent open competitions at ICDAR and ICFHR conferences between 2014 and 2017, this paper introduces four HTR benchmarks in order of increasing complexity from several points of view. For each benchmark, a specific system is proposed which overcomes results published so far under comparable conditions. Therefore, this paper establishes new state of the art baseline systems and results which aim at becoming new challenges that would hopefully drive further improvement of HTR technologies. Both the datasets and the software tools used to implement the baseline systems are made freely accessible for research purposes. [ABSTRACT FROM AUTHOR]

• Multiple particle tracking is challenging in synchrotron X-ray images. • Discriminative appearance and neighbouring topology learning are proposed. • A detection recovery method using multi-frame association is proposed. • Our method achieves tracking errors of 7.03% and association errors of 7.22%. Recent research has introduced a novel method of directly monitoring the effects of potential therapies for Cystic Fibrosis (CF) airway disease by quantifying mucociliary transit (MCT). In this method, micron-sized spherical particles are deposited into rodent airways, and synchrotron X-ray images are obtained to quantify the motion of the particles. However, the accurate tracking of these particles is challenging due to low contrast, image noise, and the presence of overlapping particles. Therefore, this paper proposes a novel method for detecting and tracking circular particles and measuring their dynamics. Accurate particle detection is achieved by applying a convolutional neural network (CNN). For robust multi-object tracking, this paper proposes a confidence model utilizing appearance and neighbouring topology learned by linear discriminant analysis. We also propose a detection recovery method using multi-frame association to restore the missed particles due to overlapping. The proposed method is tested with several different datasets and shows high levels of detection and tracking accuracy. Finally, by offering visual tracking analyses that display merging and splitting events, the proposed method can provide a better understanding of airway MCT behaviour. [ABSTRACT FROM AUTHOR]

Highlights • A novel framework termed joint graph optimization and projection learning (JGOPL) is proposed for graph-based dimensionality reduction. • The l 21 -norm based distance measurement is adopted in the loss function of our JGOPL so that its robustness to the negative influence caused by the outliers or variations of data can be improved. • In order to well exploit and preserve the local structure information of high-dimensional data, a locality constraint is introduced into the proposed JGOPL to discourage a sample from connecting with the distant samples during graph optimization. • The locality constraint and graph optimization strategy proposed is not only limited to dimensionality reduction, but also can be incorporated into other relevant graph-based tasks. Abstract Nowadays, graph-based dimensionality reduction approaches have become more and more popular due to their successful utilization for classification and clustering tasks. In these approaches, how to establish an appropriate graph is critical. To address this issue, a novel graph-based dimensionality reduction framework termed joint graph optimization and projection learning (JGOPL) is proposed in this paper. Compared with existing dimensionality reduction approaches, there are three main advantages of JGOPL. First, through performing the graph optimization and low-dimensional feature learning simultaneously, our proposed approach can accomplish the tasks of graph construction and dimensionality reduction jointly. Second, the l 21 -norm based distance measurement is adopted in the loss function of our JGOPL so that its robustness to the negative influence caused by the outliers or variations of data can be improved. Third, in order to well exploit and preserve the local structure information of high-dimensional data, a locality constraint is introduced into the proposed JGOPL to discourage a sample from connecting with the distant samples during graph optimization. Extensive classification and clustering experiments are carried out on seven publicly available databases to demonstrate the effectiveness of our approach. At last, the locality constraint and graph optimization strategy proposed in this paper is not only limited to dimensionality reduction, but also can be incorporated into other relevant graph-based tasks (such as spectral clustering). [ABSTRACT FROM AUTHOR]

Highlights • This paper proposes SQL algorithm to generate superpixel lattice. The number of labels is reduced from the number of superpixels to 4, and this small number assures efficient labeling. • Extensive experiments demonstrate that SQL outperforms the other superpixel lattice methods and is competitive with state-of-the-art methods without lattice guarantee. • SQL facilitates either MRFs or CNN to be build upon superpixels in the subsequent image analysis tasks. Abstract This paper formulates superpixel segmentation as a pixel labeling problem and proposes a quaternary labeling algorithm to generate superpixel lattice. It is achieved by seaming overlapped patches regularly placed on the image plane. Patch seaming is formulated as a pixel labeling problem, where each label indexes one patch. Once the optimal seaming is completed, all pixels covered by one retained patch constitute one superpixel. Further, four kinds of patches are distinguished and assembled into four layers correspondingly, and the patch indexes are mapped to the quaternary layer indexes. It significantly reduces the number of labels and greatly improves labelling efficiency. Furthermore, an objective function is developed to achieve optimal segmentation. Lattice structure is guaranteed by fixing patch centers to be superpixel centers, compact superpixels are assured by horizontal and vertical constraints enforced on the smooth terms, and coherent superpixels are achieved by iteratively refining the data terms. Extensive experiments on BSDS data set demonstrate that SQL algorithm significantly improves labeling efficiency, outperforms the other superpixel lattice methods, and is competitive with state-of-the-art methods without lattice guarantee. Superpixel lattice allows contextual relationships among superpixels to be easily modeled by either MRFs or CNN. [ABSTRACT FROM AUTHOR]

• Almost all these works above take the independent and identically distributed (i.i.d.) assumption, i.e., regarding all samples in various computer vision tasks as i.i.d., but they are not i.i.d. samples and the samples of many machine learning applications such as speech recognition, system diagnosis, and market forecasting are also proved to be non-i.i.d. samples. In particular, with the advent of the big data era, multi-class learning problems-such as text or image classification - can involve tens or hundreds of thousands of classes, and the samples are usually spatially-connected or temporally-correlated with their physically-connected neighbours, which can not be directly handled by these existing methods or algorithms. This implies that many samples are not satisfy the i.i.d. assumption or this assumption is very restrictive and can not be strictly justified in real-world problems. In addition, the AIO methods above are theoretically reasonable (i.e., Fisher consistent) and experiments indicate that they can lead to well-generalizing hypothesis, but efficient algorithms are lacking so far. Then three problems are posed: • Whether the AIO-MSVM algorithm based on non-i.i.d. samples is consistent? If this algorithm is consistent, what is the learning rate of this algorithm with non-i.i.d. samples? Whether the efficient AIO-MSVM algorithm can be proposed (or whether the AIO-MSVM algorithm with i.i.d. samples can be improved)? • The goal of this paper is to answer these problems above. Now, let' s highlight the main features of this paper. • The generalization bound of AIO-MSVM algorithm with u.e.M.c. samples is established and the fast learning rate is obtained. The AIO-MSVM algorithm with u.e.M.c. samples is proved to be consistent. To our knowledge, these studies are the first works on this topic. • A new algorithm, called AIO-MSVM algorithm based on q -times Markovian resampling (AIO-Mar- q), is proposed. The effectiveness and efficiency of the proposed algorithm are validated by experiments on public datasets. The generalization performance of "All-in-one" Multi-class SVM (AIO-MSVM) based on uniformly ergodic Markovian chain (u.e.M.c.) samples is considered. We establish the fast learning rate of AIO-MSVM algorithm with u.e.M.c. samples and prove that AIO-MSVM algorithm with u.e.M.c. samples is consistent. We also propose a novel AIO-MSVM algorithm based on q -times Markovian resampling (AIO-MSVM-MR), and show the numerical investigation on the learning performance of AIO-MSVM-MR based on public datasets. The experimental studies indicate that compared to the classical AIO-MSVM algorithm and other MSVM algorithms, the proposed AIO-MSVM-MR algorithm has not only smaller misclassification rate, but also less sampling and training total time. We present some discussions on the case of unbalanced training samples, the choices of q and two technical parameters, and present some explanations on the learning performance of the proposed algorithm. [ABSTRACT FROM AUTHOR]

• A novel triplet teaching graph contrastive network is proposed. Its focus on the inherent link between augmented views broadens the idea of employing augmented views. • A self-evolving adaptive augmented view generating scheme is proposed. • A stochastic hybrid module for mining missing information in different contrastive concepts is proposed. The sample distribution in the contrastive space is further improved. Unsupervised graph contrastive learning has recently emerged as the solution to the crisis of label information scarcity for graph data in the real world. However, from the general paradigm of graph contrastive learning, most of the existing methods are still flawed in the design and use of augmented views and the design of contrastive targets. Therefore, the works on how to generate reasonable augmented views and utilize them canonically and how to construct efficient and comprehensive contrastive objectives are very meaningful. Based on the teaching concept, this paper proposes a new triplet teaching graph contrastive network with self-evolving adaptive augmentation. Firstly, after carefully analyzing the internal relationships between different augmented perspectives, we present a triple teaching graph neural network framework based on the improved triplet idea. It creates contrastive objectives depending on different contrastive angle levels, providing thorough guidance for graph encoders. Secondly, a self-evolving adaptive graph augmentation scheme based on topology and feature information is proposed. It is worth mentioning that with the continuous deepening of the training process, the scheme can utilize the learnable self-attention mechanism to constantly supply our network framework with an increasing number of reliable augmented views as input. Finally, when designing the contrastive objectives, we introduce a stochastic hybrid module to mine the unexploited information, which opportunely complements the contrastive sample space formed by our network framework. Furthermore, extensive experiments on multiple real-world node classification datasets demonstrate that our model can generate better-quality node embedding for downstream tasks. The implementation of this paper is available at https://github.com/PaperMiao/T-GCSA. [ABSTRACT FROM AUTHOR]

• Person Re-Identification in a multi-camera network with non-overlapping FOVs. • Open challenges in Person Re-ID are identified and discussed. • Challenge-specific survey of 230+ papers between the years of 2015-22. • Critical analysis of current SOTA with a focus on advancements towards solutions. Person Re-Identification (Re-ID) is a critical aspect of visual surveillance systems, which aims to automatically recognize and locate individuals across a multi-camera network with non-overlapping fields-of-view. Despite significant progress in recent years through the use of deep learning-based approaches, there remain many vision-related challenges, such as occlusion, pose, background clutter, misalignment, scale, viewpoint, low resolution & illumination, and cross-domain generalization across camera modalities, that hinder the accurate identification of individuals. The majority of the proposed approaches directly or indirectly aim to solve one or multiple of these existing challenges. To further advance the development of Re-ID solutions, a comprehensive review of the current approaches is necessary. However, no focused review currently exists that analyses and highlights specific aspects for further development. To fill this gap, we present a systematic challenge-specific literature survey of about 300 papers published between 2015 and 2022, which reviews Re-ID approaches from a solution-oriented perspective. This survey is the first of its kind to provide an in-depth analysis of the different approaches used to address the various challenges in Re-ID. Furthermore, our review highlights several prominent and diverse research trends in the Re-ID domain. These trends offer a visionary perspective regarding ongoing person Re-ID research, and they may eventually lead to the development of practical real-world solutions. We highlighted the AI ethics that must be followed while developing a Re-ID solution, and recently being practiced as well. Another exciting future dimension of person Re-ID research is the long-term Re-ID, which is still under evolution. Overall, our survey aims to serve as a valuable resource for researchers and practitioners working in the field of Re-ID and to inspire the development of innovative and effective Re-ID solutions. [ABSTRACT FROM AUTHOR]

• We propose herein, a new cancelable biometric scheme based on Local Dissimilarity Map and RetinexGAN model, in order to address the issue of template protection. • An efficient way for finger vein features representation, which investigates the relationships and correlation inter and intra classes, while effectively coming up with the accidental shifts/rotations caused by the arbitrary position of the finger during image acquisition, is presented in this work. • An efficient way to deal with low light image enhancement purposes based on Retinex theory and Generative Adversarial Network(GAN), is proposed in this paper. • An efficient way to establish the best compromise between workload reduction and biometric template protection is proposed in this paper. In this paper, we propose a new biometric template protection scheme, which can deal with the finger vein biometric security threats, through using the LDM and RetinexGAN model. The RetinexGAN model is mainly used to handle the illumination and low contrast problems effectively, while efficiently extracting discriminative features from the finger vein images. The projection of extracted features into dissimilarity space is done using Local Dissimilarity Map (LDM). LDM is an efficient way for finger vein features representation, which investigates the relationships and correlation inter and intra classes, while effectively coming up with the accidental shifts/rotations caused by the arbitrary position of the finger during image acquisition. The proposed approach is successfully evaluated in terms of non-invertibility, non-linkability, revocability and performances. Experimental results and comparison analysis with the state of arts methods confirm that the proposed framework can achieve promising results. [ABSTRACT FROM AUTHOR]

• A deep reinforcement learning based framework is proposed to segment single frame image pixel-wise. • We propose a slider item combined state and a group of morphology transforming actions in the reinforcement learning segment networks. • We proved that the proposed network has strong expansibility for semi-supervised training and it is more flexible for input image size. • An identification algorithm is proposed in this paper, which provides a continuous environment for the implementation of reinforcement learning process. Automatic segmentation of the left ventricle (LV) in four-chamber view images is critical for computer-aided cardiac disease diagnosis. The complex structure of the cardiac image and the encoder-decoder networks may cause coarse segmentation results. High-accuracy LV segmentation is still a challenge with existing automatic LV segmentation methods. In this paper, we propose a slide deep reinforcement learning segmentation network for pixelwise LV segmentation. The main architecture of the slide reinforcement learning networks consists of a slider item combined state, a group of morphology transforming actions and an agent network. The specifically designed reinforcement learning state comprises an image item and a slider item, which contains both original image information and network act information. The reinforcement learning actions proposed in this paper enable accurate and fast formulation of the binary segment result for each frame by controlling the length and location of the slider. Additionally, the confidence branch proposed in our experiment provides a continuous frame series environment, and the identification algorithm avoids losing the segmentation target. The segmentation result reveals that the proposed method outperforms FCN, SegNet, U-Net and TransUnet. The IoU improved by 23.01 % , 15.4 % , 11.24 % and 6.9 %. Additionally, we demonstrate how the proposed method can be used as a semisupervised method, which is more convenient for the image annotation process. [ABSTRACT FROM AUTHOR]

• The paper proposes the Presumably Correct Decision Sets for handling uncertainty. • Similarity neighborhoods define weak, borderline and strong members of each class. • Presumably correct and incorrect sets allow detecting difficult instances. • Numerical simulations illustrate the advantages of our model compared to rough sets. The paper presents the presumably correct decision sets as a tool to analyze uncertainty in the form of inconsistency in decision systems. As a first step, problem instances are gathered into three regions containing weak members, borderline members, and strong members. This is accomplished by using the membership degrees of instances to their neighborhoods while neglecting their actual labels. As a second step, we derive the presumably correct and incorrect sets by contrasting the decision classes determined by a neighborhood function with the actual decision classes. We extract these sets from either the regions containing strong members or the whole universe, which defines the strict and relaxed versions of our theoretical formalism. These sets allow isolating the instances difficult to handle by machine learning algorithms as they are responsible for inconsistent patterns. The simulations using synthetic and real-world datasets illustrate the advantages of our model compared to rough sets, which is deemed a solid state-of-the-art approach to cope with inconsistency. In particular, it is shown that we can increase the accuracy of selected classifiers up to 36% by weighting the presumably correct and incorrect instances during the training process. [ABSTRACT FROM AUTHOR]

• We propose an improved differential evolution (DE) algorithm by composite binary crossover and Pareto refining operations to the DE. • DE solves the quantifying fraudulent transactions problem by minimizing the selected feature cost while maximizing the quantification ability. • We show that the proposed DE outperforms four other variants of DE and five state-of-the-art EAs on various credit datasets. • The results are also statically validated through the Friedman test and Wilcoxon post-hoc analysis. [Display omitted] Identification of fraudulent credit card transactions is a complex problem mainly due to the following factors: 1) The relative behavior of customers and fraudsters may alter over time. 2) The ratio of legitimate to fraudulent transactions is highly imbalanced, and 3) Investigators examine a small segment of transactions in a reasonable time frame. Researchers have proposed various algorithms to identify potential fraud in a new incoming transaction. However, these approaches require significant human investigator effort and are sometimes misleading. To address this issue, this paper proposes an improved multiobjective differential evolution (DE) algorithm to estimate the distribution of fraudulent transactions in a set of new incoming transactions, referred to as quantifying fraudulent transactions. Our paper has three major novelties. First, we present the problem formulation of cost-based feature selection with maximum quantification ability. Second, we improve the DE by applying effective trial vector generation algorithms to the random control parameter settings to exploit the advantage of individual DE variants. Third, we develop the maximum-relevancy-minimum-redundancy-based Pareto refining operator to enhance the self-learning ability of individuals in Pareto solutions. We compare our approach against four other modifications of DE and five state-of-the-art evolutionary algorithms on real-time credit datasets in streaming and non-streaming frameworks using hyper-volume, two-set coverage, and spread performance metrics. [ABSTRACT FROM AUTHOR]

• Features of the negative input image are utilized in a blind Gaussian network for the first time to give emphasis on the low contrast regions of the image. • A new multi-scale pixel attention (MSPA) block is proposed that, after analyzing the features of the negative input image at five different scales, combines the most important features to generate a feature map. • A new multi-scale feature extraction (MSFE) block is proposed, that generates a feature map after analyzing the features of the input noisy image at five different scales. • The proposed network is a lightweight and fast denoising network that takes fewer parameters and has fast run-time than many state-of-the-art blind denoising methods. In this paper, we propose a blind Gaussian denoising network that utilize the features of the input image and its negative for generating denoised output of the same. The proposed network is a dual path model which employs a multi-scale pixel attention (MSPA) block on one path and a multi-scale feature extraction (MSFE) block on another. The concept of using the features of a negative image (that it highlights the low contrast region) in blind Gaussian denoising network is, to the best of our knowledge, a first such attempt. The proposed MSPA and MSFE blocks are designed to focus on the features of the image at multiple scales. The MSPA block focuses on the important features of the negative of the input image whereas the MSFE block focuses on extracting features of the input noisy image. The features of both the images are then combined and a final residual noise is obtained, subtracting which from the input noisy image produces the final denoised result. The proposed network is lightweight and fast, due to the low number of convolutional layers involved, and produces superior results (both quantitatively and qualitatively) when compared with various traditional and learning based blind Gaussian denoising techniques. The code of this paper can be downloaded from https://github.com/RTSIR/NIFBGDNet. [ABSTRACT FROM AUTHOR]

• Deep multi-task networks have high uncertainty despite encouraging performance. • Multi-task learning system performance is sensitive to weighting strategy for tasks. • Paper proposes a novel weighting strategy to improve model uncertainty and accuracy. • This strategy enhances quality of weights by Bayesian estimator and particle filter. • Proposed tasks' weights make model have low uncertainty and high performance. Recently multi-task learning (MTL) has been widely used in different applications to build more robust models by sharing knowledge across several related tasks. However, one challenge that arises is the variability in the learning pace of different tasks causing the inefficiency of naively training all tasks. Therefore, it is of great importance to consider some coefficients to balance tasks in the process of learning, but, due to the large search space and the significance of setting them properly, conventional search methods such as grid or random search are no longer effective. In this paper, we propose a learning mechanism for these coefficients based on the high efficiency of the particle filter (PF) algorithm to deal with nonlinear search problems. PF considers each state of the tasks' coefficients as a particle and recursively converges coefficients to an optimum point. While in most previous works coefficients were evaluated to only increase performance, to address the recent concerns related to applying AI in real-world applications, we also incorporate uncertainty alongside our method to prevent learning coefficients leading to unstable outcomes. This mechanism is independent of the models main learning process and can be easily added to every learning system without changing its training algorithm. Extensive experiments on real-world data sets demonstrate the superiority of the proposed method over the state-of-the-art methods on both performance and uncertainty. We also proved the acceptable performance of the method using Cramer Rao lower bound theory. [ABSTRACT FROM AUTHOR]

• A new dataset is introduced for human action recognition in aerial videos. • Proposed dataset is annotated with 13 different actions with a total of 55,477 frames and almost 400,000 annotations. • A modified ResNeXt101 (M-ResNext101) is proposed for recognizing the human actions from aerial videos. Human actions being diverse in nature cannot be generalized, thus making it quite difficult to train a machine to recognize such diversified actions. This challenge is further compounded by the lack of availability of datasets for aerial surveillance, as collecting and annotating a large dataset is a formidable task. This paper aims to solve the problem of data scarcity by introducing a new dataset, Aeriform in-action for recognizing human actions from aerial videos. The proposed dataset consists of 32 high-resolution videos containing 13 action classes with 55,477 frames (without augmentation) and almost 400,000 annotations. It includes complex and aggressive actions such as kicking and punching, as well as drone signaling actions like waving and handshaking. The dataset also includes human-object interactions like carrying and reading. In addition to the dataset, this paper also presents a two-step deep learning framework for recognizing human actions based on the integration of human detection and action recognition module. The action recognition module adopts a modified version of the ResNeXt101 architecture (M-ResNext101) to recognize human actions in aerial videos. The performance of the proposed M-ResNext101 model is compared with 13 other deep learning models, and it outperforms all of them with an accuracy of 76.44% on the test data. The proposed dataset for human action recognition in aerial videos is available on https://surbhi-31.github.io/Aeriform-in-action/. [ABSTRACT FROM AUTHOR]