Search

3D human motion prediction, predicting future poses from a given sequence, is an issue of great significance and challenge in computer vision and machine intelligence, which can help machines in understanding human behaviors. Due to the increasing development and understanding of Deep Neural Networks (DNNs) and the availability of large-scale human motion datasets, the human motion prediction has been remarkably advanced with a surge of interest among academia and industrial community. In this context, a comprehensive survey on 3D human motion prediction is conducted for the purpose of retrospecting and analyzing relevant works from existing released literature. In addition, a pertinent taxonomy is constructed to categorize these existing approaches for 3D human motion prediction. In this survey, relevant methods are categorized into three categories: human pose representation , network structure design , and prediction target. We systematically review all relevant journal and conference papers in the field of human motion prediction since 2015, which are presented in detail based on proposed categorizations in this survey. Furthermore, the outline for the public benchmark datasets, evaluation criteria, and performance comparisons are respectively presented in this paper. The limitations of the state-of-the-art methods are discussed as well, hoping for paving the way for future explorations. [ABSTRACT FROM AUTHOR]

Frequent state monitoring and controller updates can enhance the precision of tracking control, while simultaneously overburdening the communication network transmission. In this paper, For the purpose of saving communication costs, we propose event-triggered control algorithms for the optimal tracking control problem. First, we reconstruct the discrete-time nonlinear system into a converted system. Then, the adaptive dynamic programming algorithm is employed to find the optimal controller off-line, and the event-triggered scheme is used to reduce the communication costs online. Novel triggering conditions with fewer assumptions are designed to implement the event-triggered scheme. Different from existing works, the event-triggered scheme can be introduced not only into the converted system but also into the actual system, which is more practical because the actual controller is what one can only access in practice. In addition, with the developed algorithms, the tracking error can be proved to be stable at the origin, in other words, the actual system can be guaranteed to track the desired trajectory. Algorithms developed in this paper are implemented by three neural networks, the model network, the action network and the critic network. Finally, examples are presented to verify the effectiveness and rationality of the algorithms. [ABSTRACT FROM AUTHOR]

• This is a comprehensive review of Transformer for Skeleton-based Action Recognition. • This paper proposes a new taxonomy of transformer-style techniques. • This survey aims to help researchers systematically select promising future directions. Skeleton-based action recognition has rapidly become one of the most popular and essential research topics in computer vision. The task is to analyze the characteristics of human joints and accurately classify their behaviors through deep learning technology. Skeleton provides numerous unique advantages over other data modalities, such as robustness, compactness, noise immunity, etc. In particular, the skeleton modality is extremely lightweight, which is especially beneficial for deep learning research in low-resource environments. Due to the non-European nature of skeleton data, Graph Convolution Network (GCN) has become mainstream in the past few years, leveraging the benefits of processing topological information. However, with the explosive development of transformer methods in natural language processing and computer vision, many works have applied transformer into the field of skeleton action recognition, breaking the accuracy monopoly of GCN. Therefore, we conduct a survey using transformer method for skeleton-based action recognition, forming of a taxonomy on existing works. This paper gives a comprehensive overview of the recent transformer techniques for skeleton action recognition, proposes a taxonomy of transformer-style techniques for action recognition, conducts a detailed study on benchmark datasets, compares the algorithm accuracy of standard methods, and finally discusses the future research directions and trends. To the best of our knowledge, this study is the first to describe skeleton-based action recognition techniques in the style of transformers and to suggest novel recognition taxonomies in a review. We are confident that Transformer-based action recognition technology will become mainstream in the near future, so this survey aims to help researchers systematically learn core tasks, select appropriate datasets, understand current challenges, and select promising future directions. [ABSTRACT FROM AUTHOR]

Automatic identity recognition from ear images is an active research topic in the biometric community. The ability to secretly acquire images of the ear remotely and the stability of the ear shape over time make this technology a promising alternative for surveillance, authentication, and forensic applications. In recent years, significant research has been conducted in this area. Nevertheless, challenges remain that limit the commercial use of this technology. Several phases of the ear recognition system have been studied in the literature, from ear detection, normalization, and feature extraction to classification. This paper reviews the most recent methods used to describe and classify biometric features of the ear. We propose a first taxonomy to group existing approaches to ear recognition, including 2D, 3D, and combined 2D and 3D methods, as well as an overview of historical advances in this field. It is well known that data and algorithms are the essential components in biometrics, particularly in-ear recognition. However, early ear recognition datasets were very limited and collected in laboratory with controlled environments. With the wider use of deep neural networks, a considerable amount of training data has become necessary if acceptable ear recognition performance is to be achieved. As a consequence, current ear recognition datasets have increased significantly in size. This paper gives an overview of the chronological evolution of ear recognition datasets and compares the performance of conventional vs. deep learning methods on several datasets. We proposed a second taxonomy to classify the existing databases, including 2D, 3D, and video ear datasets. Finally, some open challenges and trends are debated for future research. [ABSTRACT FROM AUTHOR]

• A linearly combined form of the objective is reformulated to control the trade-off between the return and the divergence more effectively. • An improved proximal policy optimization method (i.e., alphaPPO) is proposed, with a more elaborative alpha divergence for two adjacent policies. • The effectiveness of our alphaPPO is validated using detailed experimental comparison and analysis for six benchmark environments. Proximal policy optimization (PPO) is a recent advancement in reinforcement learning, which is formulated as an unconstrained optimization problem including two terms: accumulative discount return and Kullback–Leibler (KL) divergence. Currently, there are three PPO versions: primary, adaptive, and clipping. The most widely used PPO algorithm is the clipping version, in which the KL divergence is replaced by a clipping function to measure the difference between two policies indirectly. In this paper, we revisit this primary PPO and improve it in two aspects. One is to reformulate it as a linearly combined form to control the trade-off between two terms. The other is to substitute a parametric alpha divergence for KL divergence to measure the difference of two policies more effectively. This novel PPO variant is referred to as alphaPPO in this paper. Experiments on six benchmark environments verify the effectiveness of our alphaPPO, compared with clipping and combined PPOs. [ABSTRACT FROM AUTHOR]

• A novel global optimization algorithm, rime optimization algorithm (RIME), is proposed. • RIME simulates the growth and crossover behavior of the rime-particle population. • The performance of RIME is adequately discussed based on benchmark functions. • RIME has better convergence accuracy and speed compared to other methods. • RIME is applied to solving five practical engineering optimization problems. This paper proposes an efficient optimization algorithm based on the physical phenomenon of rime-ice, called the RIME. The RIME algorithm implements the exploration and exploitation behaviors in the optimization methods by simulating the soft-rime and hard-rime growth process of rime-ice and constructing a soft-rime search strategy and a hard-rime puncture mechanism. Meanwhile, the greedy selection mechanism in the algorithm is improved, and the population is updated in the stage of selecting the optimal solution to enhance the exploitation capability of the RIME. In the experimental, this paper conducts qualitative analysis experiments on the RIME to clarify the characteristics of the algorithm in the process of finding the optimal solution. The performance of RIME is then tested on a total of 42 functions in the classic IEEE CEC2017 and the latest IEEE CEC2022 test sets. The proposed algorithm is compared with 10 well-established algorithms and 10 latest improved algorithms to verify its performance advantage. In addition, this paper designs experiments for the parametric analysis of RIME to discuss the potential of the algorithm in running different parameters and handling different problems. Finally, this paper applies RIME to five practical engineering problems to verify its effectiveness and superiority in real-world problems. The statistical and comparison results show that the RIME is a strong and competitive algorithm. The source code of the RIME 1 1 Source codes of RIME algorithm are also available at: https://codeocean.com/capsule/4472937/tree and https://www.mathworks.com/matlabcentral/fileexchange/124610-rime-a-physics-based-optimization and https://github.com/aliasgharheidaricom/RIME-A-physics-based-optimization. algorithm and associated files are publicly accessible at https://aliasgharheidari.com/RIME.html. [ABSTRACT FROM AUTHOR]

Conventional convergence analysis on mini-batch learning is usually based on the stochastic gradient concept, in which we assume that the training data are presented in a random order. Also, some convergence results require that the learning rate should decrease with the number of training cycles, and that the objective function is a smooth function. Practically speaking, a deterministic presentation scheme with a fixed learning rate is more preferable. Hence, there is a gap between theoretical results and actual implementation. This paper aims at filling the gap. We use the radial basis function (RBF) model for nonlinear regression problems as an example to analyze the convergence properties of mini-batch learning. This paper considers a nonsmooth objective function, which consists of three terms. The coexistence of these three terms is able to handle a number of situations. The first term is a conventional training set error. The second term is a quadratic term which is used to suppress the effect of imperfections in the implementation. The last term is an ℓ 1 -norm term which is used to select important RBF nodes for the resultant network. Note that the ℓ 1 -norm term is a nonsmooth function. Although a nonsmooth ℓ 1 -norm is included and the mini-batch algorithm is deterministic, we are still able to derive the convergence properties, including the sufficient conditions for convergence and range of learning rate. With our results, we have a better theoretical understanding on the behaviour of mini-batch learning and obtain some guidelines on choosing the learning rate. The analysis results can be extended to other flat structural neural network models and other objective functions, which are with quadratic terms and ℓ 1 -norm. [ABSTRACT FROM AUTHOR]

This paper tackles the event-triggering filtering problem for a class of nonlinear networked control systems subject to the prescribed disturbance attenuation level. To alleviate the data transmission burden of communication channels, an event-triggering mechanism is adopted, where the data can be transmitted only if certain condition is fulfilled. The T-S fuzzy model approach is utilized to approximate the nonlinear dynamics. It is the purpose of this paper to design a filter for the addressed nonlinear system ensuring that the filtering error system is asymptotically stable whereas the prespecified H ∞ criterion is guaranteed. By using Lyapunov approach in combination with matrix analysis method, the sufficient conditions are established for the existence of desired filtering parameters in terms of the solvability of certain linear matrix inequalities. Finally, a numerical simulation example is put forward to demonstrate the validity and effectiveness of the developed theoretical results. [ABSTRACT FROM AUTHOR]

Numerous visual domain adaptation methods have been proposed for transferring knowledge from a well-labeled source domain to an unlabeled but related target domain. Most of existing works are only geared to closed set domain adaptation, where an identical label space is shared between two domains. In this paper, we focus on a more realistic but challenging scenario, open set domain adaptation, where the target domain contains unknown classes that do not appear in the label space of source domain. The main task of open set domain adaptation is to simultaneously recognize the target images of known classes and those of unknown classes correctly. To achieve this goal, in this paper, we propose a novel open set domain adaptation method, which consists of two parts: latent structure discovery and kernelized classifier learning. In the first part, we employ an adaptive discriminative graph learning strategy to capture the intrinsic manifold structure of the source and target domain data in the latent feature space, such that the boundaries among all classes will be delineated more clearly. In the second part, the samples from the latent feature space are mapped into a high-dimensional kernel space to make them linearly separable, and a linear classifier is learned by jointly operating unknown target samples separating, known samples matching and local structure preserving. As the optimization problem is not convex with all variables, we devise an efficient iterative algorithm to solve it. The extensive experimental results on five image datasets confirm the superiority of the proposed method compared with the state-of-the-art traditional and deep competitors. [ABSTRACT FROM AUTHOR]

The increasing availability of remote sensing data allows dealing with spatial-spectral limitations by means of pan-sharpening methods. However, fusing inter-sensor data poses important challenges, in terms of resolution differences, sensor-dependent deformations and ground-truth data availability, that demand more accurate pan-sharpening solutions. In response, this paper proposes a novel deep learning-based pan-sharpening model which is termed as the double-U network for self-supervised pan-sharpening (W-NetPan). In more details, the proposed architecture adopts an innovative W-shape that integrates two U-Net segments which sequentially work for spatially matching and fusing inter-sensor multi-modal data. In this way, a synergic effect is produced where the first segment resolves inter-sensor deviations while stimulating the second one to achieve a more accurate data fusion. Additionally, a joint loss formulation is proposed for effectively training the proposed model without external data supervision. The experimental comparison, conducted over four coupled Sentinel-2 and Sentinel-3 datasets, reveals the advantages of W-NetPan with respect to several of the most important state-of-the-art pan-sharpening methods available in the literature. The codes related to this paper will be available at https://github.com/rufernan/WNetPan. [ABSTRACT FROM AUTHOR]

• We focus on the data dilemma encountered in the field of human activity understanding, solve practical application problems from a new perspective, and use the proposed method to reduce the model's dependence on data. • We construct a human physical activity dataset containing 10 categories Human SA-10 for use in human activity understanding research. • A Core-Weight Entropy data information evaluation method that can be applied to human behavior recognition tasks is proposed. On Human SA-10, our method can reduce data usage by 50%. Compared to other methods, this method achieved state-of-the-art performance. • In addition, we propose a new unlabeled data redundancy information removal module, which effectively avoids introducing similar data into the training set. With the development of computer vision, the research on human activity understanding has been greatly promoted. The recognition algorithm based on vision transformer has made some achievements in a large number of computer vision tasks, but it still needs to be driven by a large amount of data. How to get rid of the constraints of large amounts of data is crucial for human behavior recognition based on vision transformer. This paper focuses on solving the dilemma of big data, and tries to achieve a high-performance model through a small amount of high information human activity data. The advantage of our work is that by studying feature distribution, we proposed a core weight entropy data information evaluation method for obtaining high information data, and through redundant information elimination strategy, we can avoid introducing similar data. A large number of experimental results show the effectiveness of the proposed method. Compared with existing methods, our method reduces the data consumption by 5% to 30%, and can achieve the performance of using only 50% of 100% data. More importantly, the data our method selected has no redundancy, which is not available in other methods. In addition, we carried out a large number of ablation experiments to prove the rationality of the method. The work of this paper solves the challenge of relying on a large amount of data when using the visual converter to recognize human behavior, which is of practical significance for realizing efficient human activity understanding research with low data. [ABSTRACT FROM AUTHOR]

This paper applies a terminal learning strategy to study distributed quadratic optimization problems. Since the optimal state is unknown in advance, the tracking error information is generally unavailable. To achieve the optimal state without the tracking error information, the terminal consensus iterative learning scheme is used to solve the problem. And the terminal consensus state is obtained without the global information of network. On this basis, the optimal target is also achieved by choosing the proper initial state and learning parameters. And the optimization problem is studied with the constraints of state and control input. Results show that our approach is effective. Compared with existing distributed optimization methods, the learning strategy in this paper provides another effective analysis scheme. Last, a numerical example is presented to show the effective aspects of the method. [ABSTRACT FROM AUTHOR]

[Display omitted] • The dense feature adaptive connection module improves the quality of depth map. • The distributed 3D convolution reduces the computational cost and memory space. • The joint loss function makes the network sensitive to small depth structures. The goal of Multi-View Stereo is to reconstruct the 3D point cloud model from multiple views. With the development of deep learning, more and more learning-based research has achieved remarkable results. However, existing methods ignore the fine-grained features of the bottom layer, which leads to the poor quality of model reconstruction, especially in terms of completeness. Besides, current methods still rely on a large amount of consumed memory resources because of the application of 3D convolution. To this end, this paper proposes a Multiple Granularities Feature Fusion Network for Multi-View Stereo, an end-to-end depth estimation network combining global and local features, which is characterized by fine-granularity multi-feature fusion. Firstly, we propose a dense feature adaptive connection module, which can adaptively fuse the global and local features in the scene, provide a more complete and effective feature map for inferring a more detailed depth map, and make the ultimate model more complete. Secondly, in order to further improve the accuracy and completeness of the reconstructed point cloud, we introduce normal and edge loss futead of only using depth loss functions as in the existing methods, which makes the network more sensitive to small depth structures. Finally, we propose distributed 3D convolution instead of traditional 3D convolution, which reduces memory consumption. The experimental results on the DTU and Tanks & Temples datasets demonstrate that the proposed method in this papaer achieves the state-of-the-art performance, which proves the accuracy and effectiveness of the MG-MVSNet proposed in this paper. [ABSTRACT FROM AUTHOR]

This paper investigates the synchronization of complex dynamic networks with time-varying delay and general semi-Markovian jump. The general transition rates include completely unknown and uncertain but bounded as two special cases. First, by introducing auxiliary vectors with a few nonorthogonal polynomials, two free-matrix-based integral inequalities are developed, which encompass some existing ones as special cases. Second, an integral- based delay-product-type Lyapunov-Krasovskii functional is constructed, which fully considers the information of time delay. By utilizing a deley-dependent controller, two sufficient conditions are derived to realize the global stochastic mean-square synchronization by employing the established inequalities to evaluate the infinitesimal generator of the functional. This paper takes all possibilities into consideration and divides the general transition rates into five cases, which is never investigated before. Finally a numerical example is given to show the effectiveness and practicality of the presented method. [ABSTRACT FROM AUTHOR]

• Providing an in-depth and comprehensive review related to underwater object detection. • Extensively exploring the correlation between underwater image enhancement and underwater object detection. • Analyzing the impact and augmentation forms of underwater image enhancement in underwater object detection. • Discussing the challenges, future trends and applications of underwater object detection. Underwater object detection is one of the most challenging research topics in computer vision technology. The complex underwater environment makes underwater images suffer from high noise, low visibility, blurred edges, low contrast and color deviation, which brings significant challenges to underwater object detection tasks. In underwater object detection tasks, traditional object detection methods often perform poorly in terms of accuracy and generalization capabilities. Underwater object detection requires accurate, stable, generalizable, real-time and lightweight detection models, for which many researchers have proposed various underwater object detection techniques based on deep learning. Although many outstanding results have been achieved on underwater object detection over the years, the research status of underwater object detection techniques are still lack of unified induction, and some existing problems need to be further probed from the latest perspective. In addition, previous reviews lack analysis on the relationship between underwater image enhancement and object detection. Therefore, this paper provides a comprehensive review of the current research challenges, future development trends, and potential applications of underwater object detection techniques. More importantly, this paper has explored the internal relationship between underwater image enhancement and object detection, and analyzed the possible implementation manners of underwater image enhancement in the object detection task in order to further enhance its benefits. The experiments show the performances of current underwater image enhancement and state-of-the-art object detection algorithms, point out their limitations, and indicate that there is not a strict positive correlation between underwater image enhancement and the accuracy improvement of object detection. The domain shift caused by underwater image enhancement cannot be ignored. This paper can be regarded as a guide for future works on underwater object detection. [ABSTRACT FROM AUTHOR]

Generative Adversarial Networks (GANs) are well-known tools for data generation and semi-supervised classification. GANs, with less labeled data, outperform Deep Neural Networks (DNNs) and Convolutional Neural Networks (CNNs) in classification tasks. The success of GANs in classification tasks motivates the development of GAN-based techniques for semi-supervised regression tasks. However, developing GANs for regression introduces two major challenges: (1) inherent instability in the GAN formulation and (2) performing regression and achieving stability simultaneously. This paper introduces techniques that show improvement in the GANs' regression capability. We bake a differentiable fuzzy logic system at multiple locations in a GAN. The fuzzy logic takes the output of either the generator or the discriminator to predict the output, y , and evaluate the generator's performance. We outline the results of applying the fuzzy logic system across multiple GANs and summarize each approach's efficacy. This paper shows that adding a fuzzy logic layer can enhance GAN's ability to perform regression; the most desirable injection location is problem-specific, and we show this through experiments over various datasets. Besides, we demonstrate empirically that the fuzzy-infused GANs are competitive with the DNNs. [ABSTRACT FROM AUTHOR]

• This paper proposes a novel multivariate dictionary learning framework. • A coherence enhancement term to improve the coherent representing of the coding coefficients under different resolutions. • A multivariate dictionary optimization method to solve dictionaries involving the calculation of fractional norm. • The proposed method achieves the state-of-the-art performance on several benchmark datasets. Off-the-shelf dictionary learning algorithms have achieved satisfactory results in small sample face recognition applications. However, the achieved results depend on the facial images obtained at a single resolution. In practice, the resolution of the images captured on the same target is different because of the different shooting equipment and different shooting distances. These images of the same category at different resolutions will pose a great challenge to these algorithms. In this paper, we propose a Joint Coupled Representation and Homogeneous Reconstruction (JCRHR) for multi-resolution small sample face recognition. In JCRHR, an analysis dictionary is introduced and combined with the synthetic dictionary for coupled representation learning, which better reveals the relationship between coding coefficients and samples. In addition, a coherence enhancement term is proposed to improve the coherent representation of the coding coefficients at different resolutions, which facilitates the reconstruction of the sample by its homogeneous atoms. Moreover, each sample at different resolutions is assigned a different coding coefficient in the multi-dictionary learning process, so that the learned dictionary is more in line with the actual situation. Furthermore, a regularization term based on the fractional norm is drawn into the dictionary coupled learning to remove the redundant information in the dictionary, which can reduce the negative impacts of the redundant information. Comprehensive results demonstrate that the proposed JCRHR method achieves better results than the state-of-the-art methods, on several small sample face databases. [ABSTRACT FROM AUTHOR]

• A high-performance method is proposed for RGB-D salient object detection. • RGB and depth data are adaptively fused to boost the performance. • Features of all levels are refined in an iterative manner. • The proposed method outperforms other state-of-the-art models on six datasets. Existing state-of-the-art RGB-D saliency detection models mainly utilize the depth information as complementary cues to enhance the RGB information. However, depth maps can be easily influenced by environment and hence are full of noises. Thus, indiscriminately integrating multi-modality (i.e., RGB and depth) features may induce noise-degraded saliency maps. In this paper, we propose a novel Adaptive Fusion Network (AFNet) to solve this problem. Specifically, we design a triplet encoder network consisting of three subnetworks to process RGB, depth, and fused features, respectively. The three subnetworks are interlinked and form a grid net to facilitate mutual refinement of these multi-modality features. Moreover, we propose a Multi-modality Feature Interaction (MFI) module to exploit complementary cues between depth and RGB modalities and adaptively fuse the multi-modality features. Finally, we design the Cascaded Feature Interweaved Decoder (CFID) to exploit complementary information between multi-level features and refine them iteratively to achieve accurate saliency detection. Experimental results on six commonly used benchmark datasets verify that the proposed AFNet outperforms 20 state-of-the-art counterparts in terms of six widely adopted evaluation metrics. Source code will be publicly available at https://github.com/clelouch/AFNet upon paper acceptance. [ABSTRACT FROM AUTHOR]

• This paper introduces the mutual learning strategy train the brain tumor segmentation network, using the shallowest feature map to supervise the subsequent feature map of the network. using the deepest logits to supervise the previous shallow network's logits. The shallow feature map and deep logit supervise mutually and improve the accuracy of tumor sub-region segmentation. • This paper introduces the depth supervision to train this network, using the prediction of each up-sample layer is to deep supervise the training process for enlarging the receptive field to improve the overall segmentation accuracy. • A large number of experiments on BraTS dataset show that our method can effectively improve the accuracy of brain tumor segmentation and achieve the performance of SOTA. Deep learning methods have been successfully applied to Brain tumor segmentation. However, the extreme data imbalance exists in the different sub-regions of tumor, results in training the deep learning methods on these data will reduce the accuracy of segmentation. We introduce the deep mutual learning strategy to address the challenges, the proposed integrates transformer layers in both encoder and decoder of a U-Net architecture. In the network, using the prediction of up-sampled layer is to deep supervise the training process for enlarging the receptive field to extract features, the feature map of the shallowest layer supervises the subsequent feature map of layers to keep more edge information to guide the sub-region segmentation accuracy. the classification logits of the deepest layer supervise the previous layer of logits to get more semantic information for distinguish of tumor sub-regions. Furthermore, the feature map and the classification logits supervise mutually to improve the overall segmentation accuracy. The experimental results on benchmark dataset shows that our method has significant performance gain over existing methods. [ABSTRACT FROM AUTHOR]

Neural dynamics plays an important role in handling various complex problems related to matrices or even tensors, e.g., the multi-linear M -tensor equation investigated in this paper. However, the existing methods for computing the time-dependent multi-linear M -tensor equation bear the following weaknesses: 1) all of them are under the short-time invariant hypothesis, thereby generating considerable residual errors for time-dependent ones; 2) most of them are depicted in continuous-time form, which can not be directly implemented in the digital equipment; and 3) all of them only consider the noise-free conditions, lacking robustness over truncation errors and round-off errors widely existing in the digital equipment. This paper remedies these three weaknesses by proposing a noise-suppressing discrete-time neural dynamics (NSDTND) model for the time-dependent multi-linear M -tensor equation. Additionally, analyses on the convergence and robustness are shown to demonstrate that the proposed NSDTND model is globally convergent and has a superior immunity to noises. Then, numerical experimental verifications and an application to the particle movement are provided to prove the superiority and effectiveness of the proposed NSDTND model for solving time-dependent multi-linear M -tensor equation with noises considered. [ABSTRACT FROM AUTHOR]

The training method for deep neural networks mainly adopts the gradient descent (GD) method. These methods, however, are very sensitive to initialization and hyperparameters. In this paper, an enhanced gradient descent method guided by the trajectory-based method for training deep neural networks, termed the Trajectory Unified Framework (TJU) method, is presented. From a theoretical viewpoint, the robustness of the TJU-based method is supported by an analytical basis presented in the paper. From a computational viewpoint, a TJU methodology consisting of a Block-Diagonal-Pseudo-Transient-Continuation method and a gradient descent method, termed the TJU-GD method, for training deep neural networks is added to obtain high-quality results. Furthermore, to resolve the issue of imbalanced classification, a TJU-Focal-GD method is developed and evaluated. Experimental numerical evaluation of the proposed TJU-GD on various public datasets reveals that the proposed method can achieve great improvements over baseline methods. Specifically, the proposed TJU-Focal-GD also possesses several advantages over other methods for a class of imbalanced datasets from the homemade power line inspection dataset (PLID). [ABSTRACT FROM AUTHOR]

Machine learning (ML) has been recently introduced to solving optimization problems, especially for combinatorial optimization (CO) tasks. In this paper, we survey the trend of leveraging ML to solve the mixed-integer programming problem (MIP). Theoretically, MIP is an NP-hard problem, and most CO problems can be formulated as MIP. Like other CO problems, the human-designed heuristic algorithms for MIP rely on good initial solutions and cost a lot of computational resources. Therefore, researchers consider applying machine learning methods to solve MIP since ML-enhanced approaches can provide the solution based on the typical patterns from the training data. Specifically, we first introduce the formulation and preliminaries of MIP and representative traditional solvers. Then, we show the integration of machine learning and MIP with detailed discussions on related learning-based methods, which can be further classified into exact and heuristic algorithms. Finally, we propose the outlook for learning-based MIP solvers, the direction toward more combinatorial optimization problems beyond MIP, and the mutual embrace of traditional solvers and ML components. We maintain a list of papers that utilize machine learning technologies to solve combinatorial optimization problems, which is available at https://github.com/Thinklab-SJTU/awesome-ml4co. [ABSTRACT FROM AUTHOR]

Digital video is widely used to record people's daily lives and share people's moods, but few researchers have conducted research on the consistency of emotional expression between short videos and music. In order to be able to match the appropriate background music to the short video image autonomously and efficiently, the paper analyzed the emotional connection between the two from the audio-visual synesthesia. First, emotional semantics was used as a bridge to connect video data and music data, and a video-music synesthesia data set based on semantic words was constructed. Then, an attention mechanism was incorporated to better extract key features in video images. In the extraction of music features, an improved lenet5 network was used, and the optimal network parameters were determined through experiments. Finally, the two types of features were fused and the mutual retrieval between video and music was performed. In order to compare the performance of different models, different CNN models were calculated in the processing of video images, including VGG16, VGG19, AlexNet and GoogleNet, and the attention mechanism was added to each network for calculation to compare its retrieval accuracy. In the processing of music data, different CNN algorithms were also used for comparative experiments, and networks with different layers were used to determine the optimal results. The experimental results show that the audiovisual synesthesia retrieval model based on emotion can effectively measure the emotional similarity between video images and music, and the method of the paper can produce a good match between them. The research method of the paper is the exploration of computer synesthetic intelligence, which can stimulate the creative inspiration of image and music creative designers. While enhancing the emotional experience of digital products, it also improves the efficiency and quality of development. [ABSTRACT FROM AUTHOR]

• Processing Complex-valued PolSAR Data Using Complex-valued Convolutional Neural Network (CV-CNN). • Uses a Gaussian-type activation function (GTAF) that preserves the integrity of complex-valued operations. • Introduces learnable Gaussian parameters for GTAF, and designs two multi-channel activation methods. • The classification accuracy is better than that of existing state-of-the-art methods in three datasets. To process complex-valued information such as SAR signals conveniently, the complex-valued convolutional neural network (CV-CNN) has been proposed in recent years, and it has achieved great success in SAR image recognition. This paper proposes an activation function with learnable parameters based on the Gaussian-type activation function (GTAF) in CV-CNN to improve the utilization of information in the real and imaginary parts of the neuro. For the multi-channel input of the feature map, this paper discusses two ways to set the parameters of the Gaussian-type activation function. One is that all channels share the same parameters, called the channel-sharing Gaussian-type activation function (CSGTAF). The other is that each channel has its independent parameters, called the channel-exclusive Gaussian-type activation function (CEGTAF). In addition, this paper derives the backpropagation formula of both CSGTAF and CEGTAF in detail for the training process of CV-CNN. This paper performs experimental analysis on three L-band standard PolSAR datasets. The experimental results show that, compared with the traditional method and the Gaussian activation function with fixed parameters, both CSGTAF and CEGTAF achieve higher recognition accuracy, and the difference in the recognition effect of different targets in the same dataset is little. Both show good recognition performance and have good stability and versatility. [ABSTRACT FROM AUTHOR]

• This paper presents a practical tutorial on solving optimization problems via PlatEMO, by means of abundant examples and source codes. • This paper is the first tutorial for the newest version PlatEMO v4.0. • This paper does not go deep into the technical details of algorithms, but aims to enable beginners to use PlatEMO at a low cost, which is much easier to be understood than the user manual of PlatEMO. • This paper is written according to many questions raised by users in the last five years. PlatEMO is an open-source platform for solving complex optimization problems, which provides a variety of metaheuristics including evolutionary algorithms, swarm intelligence algorithms, multi-objective optimization algorithms, surrogate-assisted optimization algorithms, and many others. Due to the problem-independent nature of most metaheuristics, they are versatile for solving problems with various difficulties such as multimodal landscapes, discrete search spaces, multiple objectives, strict constraints, and expensive evaluations, regardless of the fields the problems belong to. Since PlatEMO was published in 2017, it has been used by many researchers from both academia and industry in the computational intelligence community. However, the basic terms and concepts about optimization may confuse practitioners and junior researchers new to metaheuristics. Hence, this paper presents a practical introduction to the use of PlatEMO 4.0, focusing on the procedures of defining problems, selecting suitable metaheuristics, and collecting results. Note, however, that a description of the technical details of metaheuristics is beyond the scope of this paper and interested readers may refer to the cited references. [ABSTRACT FROM AUTHOR]

In this paper, a three-step model based on the integration of Deep Neural Networks (DNN) and Decision Models is introduced for image classification which is inspired by the human visual system. To make a decision about an object, many actions should be done in a hierarchical process in the brain. First, the retina receives visual stimuli and transfers them to the visual cortex in the brain. The information extracted in the visual cortex, is accumulated over time to select an appropriate response. Many of the current decision-making models do not show how each image is converted into useful information for the decision model. Some models have used neural networks to convert each image into the information needed in the decision-making model; however, the role of the retina is ignored among these models. In this paper, a combination of retina inspired filters, CNN-based description and accumulator-based decision model is used to classify images. This model's structure resembles the human brain due to the usage of the DoG filter bank as retina inspired filter in the first stage of it. This model shows a significant improvement in accuracy in comparison to other models; furthermore, its performance is acceptable even with the small sample training set. [ABSTRACT FROM AUTHOR]

POI(Point of Interest) recommendation is a basic and on-going issue in LBSN (Location-based Social Network) services. In this paper, a novel POI recommendation approach which is based on LBSN and multi-graph fusion is proposed. First, we take advantages of the graph neural network to construct user-POI interaction graph based on the rating data of users and construct user social graph based on the user social networks. First-order friends and high-order friends will be considered simultaneously in the user social graph. And then, we present a spectral cluster-based algorithm to gain the latent vector of the POI in location space. After this, the graph neural network is used to learn the information above. Lastly, we predict the score based on the aforementioned information and pick out the top- k POIs with the highest scores to form a recommendation list. Extensive experiments conducted on real datasets demonstrated that the method proposed in this paper can effectively generate the embedding vectors of users and POIs, and can achieve high recommendation accuracy as well. [ABSTRACT FROM AUTHOR]

• This paper aims to construct a NN DETAC scheme for interconnected nonlinear system against Dos attacks and external disturbances. • A switching-type adaptive state observer with a disturbance estimation value is proposed and an anti-disturbance decentralized event-triggered adaptive control scheme is developed. The proposed method can enhance system anti-disturbance ability. • Different from the sampled-data control scheme in the related literature, the NN DETAC scheme is developed, which can efficiently save communication resources. • By employing the properties of the hyperbolic tangent function, the interconnection terms no longer meet the global Lipschitz conditions, which relaxes the constrain condition. This paper discusses the issue of decentralized event-triggered adaptive neural network (NN) control for nonstrict-feedback nonlinear interconnected systems with external disturbances and intermittent denial-of-service (DoS) attacks. In the presence of DoS attack, all state variables are not used to design a feedback controller via the standard backstepping method. To solve this problem, a novel switching-type adaptive state observer with a disturbance compensation is constructed, where the disturbance compensation is obtained via constructing a disturbance observer. A decentralized event-triggered adaptive controller is designed by using the backstepping method to weaken the influences of DoS attack and the waste of communication resources, where a first-order sliding mode differentiator is introduced to prevent the "calculation explosion". By using linear matrix inequality techniques, some solvable sufficient conditions are attained to derive the observer gain. The closed-loop system is proved to be stable via the improved average dell time method and the piecewise Lyapunov stability theories. This control scheme ensures that all closed-loop signals remain bounded. Finally, simulation results are utilized to demonstrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

For the complex dynamical network (CDN) with uncertainties, this paper defines the links synchronization and synthesizes the adaptive control scheme to realize it. Generally speaking, a CDN can be considered as the composition system with the two coupled dynamic subsystems: the nodes subsystem (NS) and the links subsystem (LS). We observed that there are many results in the existing literature on the NS synchronization, which implies that the state at each node tends to be the same when the NS synchronization happens. However, the LS synchronization is rarely discussed in the existing literature due to its unknown meaning and engineering practice background. Inspired by the ego-networks, this paper employs the time-varying outgoing links at each node to geometrically describe the changing of network topologic structure and regards the values-weighted of outgoing links as the state variables of LS, by which the LS synchronization is introduced, the corresponding control scheme for its implementation is synthesized under the condition that the state of NS is available and the state of LS is unavailable. The control scheme includes the adaptive controller for NS and the coupling strategy for LS. Finally, the effectiveness of proposed control scheme in this paper is verified by a numerical example. [ABSTRACT FROM AUTHOR]

Nowadays, neural networks have been widely applied in many fields such as pattern recognition, signal and image processing and control theory. Over the past two decades or so, the analysis and synthesis for neural networks have received significant research attention. This paper provides a survey on the analysis and synthesis for neural networks, which is mainly concerned with the recent advances on stability analysis, state estimation and synchronization control for neural networks. First of all, the paper summarizes the recent results on the stability analysis for delayed neural networks, especially for neural networks with multiple discrete delays, neural networks with distributed delays, and discrete-time delayed neural networks. Then, the paper reviews the recent advances regarding the state estimation for neural networks with the emphasis on the network-based state estimation. Subsequently, the paper provides an overview on the synchronization control for neural networks. Finally, the conclusions and further research directions are given. [ABSTRACT FROM AUTHOR]

• This paper proposes a filter pruning approach for CNN model compression by transferring the knowledge of unimportant filters to the filters of higher importance. • Before pruning unimportant filters, a custom regularizer is utilized for knowledge transfer, which increases the gap between L1-norms of important and unimportant filters. • The effect of the penalty imposed in the custom regularizer is analyzed to justify the need for knowledge transfer before pruning. • In order to validate the robustness of the proposed framework across different CNN architectures, we experiment with five popular CNNs, namely, LeNet-5, VGG-16, ResNet-56, ResNet-110, and ResNet-50. • Experiments are performed on three benchmark datasets MNIST, CIFAR-10, and ImageNet. • An improvement over the baseline in terms of accuracy is observed even after removing 95.15%, 62.28%, and 62.39% of the Floating Point OPerations (FLOPs) from architectures LeNet-5, ResNet-56, and ResNet-110, respectively. As the deep learning models have been widely used in recent years, there is a high demand for reducing the model size in terms of memory and computation without much compromise in the model performance. Filter pruning is a very widely adopted strategy for model compression. The existing filter pruning methods identify the unimportant filters and prune them without worrying about information loss. They try to recover the same by fine-tuning the remaining filters, limiting their performance. In this paper, we tackle this problem by utilizing the knowledge from unimportant filters before pruning to minimize information loss. First, the proposed method identifies the unimportant and important filters by exploiting the lower and higher importance, respectively, using the L 1 -norm of filters. Next, the proposed custom UFKT-Reg regularizer ( R ufkt ) transfers the knowledge from unimportant filters before pruning to remaining filters, notably to a fixed number of important filters. Hence, the proposed method minimizes information loss due to the removal of unimportant filters. The experiments are conducted using the three benchmark datasets, including MNIST, CIFAR-10, and ImageNet. The proposed filter pruning method outperforms many recent state-of-the-art filter pruning methods. An improvement over the baseline in terms of accuracy is observed even after removing 95.15%, 62.28%, and 62.39% of the Floating Point OPerations (FLOPs) from architectures LeNet-5, ResNet-56, and ResNet-110, respectively. After pruning 53.25% of FLOPS from ResNet-50, only 1.02% and 0.47% of drops are observed in top-1 and top-5 accuracies, respectively. The code used in this paper will be publicly available at (https://github.com/sarvanichinthapalli/UFKT). [ABSTRACT FROM AUTHOR]

Stochastic optimization acceleration is extremely significant and challenging for deep neural networks (DNNs). In recent years, several novel proportional-integral–differential-based (PID-based) optimizers have been proposed to speed up the optimization by alleviating the oscillation behavior of stochastic gradient descent with momentum (SGD-M), yet lacked theoretical analysis. Along this line of research, this paper adopts dynamical system theory to design a new hybrid optimizer and present theoretical analysis. Firstly, it is found that DNN optimization is equivalent to a discrete time dynamical system. Building upon the equivalence, high order augmented dynamical system viewpoint is utilized to design a PI-like optimizer for ensuring high accuracy, which is more stable than SGD-M. Then, hybrid dynamical system viewpoint is employed to improve the PI-like optimizer as a new hybrid form for suppressing oscillation and accelerating optimization. Lyapunov method, Taylor series, matrix theory and equilibrium are combined to theoretically investigate the convergence and the oscillation of loss function, showing that the proposed hybrid optimizer can alleviate oscillation, boost optimization speed, and maintain high accuracy. In theoretical analyses, explicit conditions of hyper-parameters that guarantee training stability are calculated and presented, practically guiding the adjustment of hyper-parameters and promoting the application of hybrid optimizer. Experiments are presented on three commonly used benchmark datasets, i.e., MNIST, CIFAR10 and CIFAR100, demonstrating that the hybrid optimizer obtains up to 42% acceleration with competitive accuracy relative to state-of-the-art optimizers. In short, this paper not only presents a new hybrid optimizer for accelerating optimization, but also provides a novel, theoretical and systematic perspective to find and analyze new optimizer for DNNs. [ABSTRACT FROM AUTHOR]

In this paper, an approach that makes use of knuckle creases and fingernails for person identification is presented. It introduces a framework for automatic person identification that includes localisation of the region of interest (ROI) of many components within hand images, recognition and segmentation of the detected components using bounding boxes, and similarity matching between two different sets of segmented images. The following hand components are considered: i) the metacarpophalangeal (MCP) joint, commonly known as the base knuckle; ii) the proximal interphalangeal (PIP) joint, commonly known as the major knuckle; iii) the distal interphalangeal (DIP) joint, commonly known as the minor knuckle; iv) the interphalangeal (IP) joint, commonly known as the thumb knuckle, and v) the fingernails. Crucial elements of the proposed framework are the feature extraction and similarity matching. This paper exploits different deep learning neural networks (DLNNs), which are essential in extracting discriminative high-level abstract features. We further use various similarity measures for the matching process. We validate the proposed approach on well-known benchmarks, including the 11k Hands dataset and the Hong Kong Polytechnic University Contactless Hand Dorsal Images known as PolyU. The results indicate that knuckle patterns and fingernails play a significant role in the person identification framework. The 11K Hands dataset results indicate that the left-hand results are better than the right-hand results and the fingernails produce consistently higher identification results than other hand components, with a rank-1 score of 100 %. In addition, the PolyU dataset attains 100 % in the fingernail of the thumb finger. [ABSTRACT FROM AUTHOR]

[Display omitted] The figure below is an overview of the proposed framework in this paper. Point clouds with xyz coordinates and RGB attributes are fed into the network, and the output includes semantic labels and instance labels. The proposed region ensemble structure includes Random Sampling-based Set Abstraction (RS-SA), Adaptive Regional Feature Complementary (ARFC), and Affinity-based Regional Relation Reasoning (AR3) modules. RS-SA denotes the random sampling-based set abstraction. The down-sampling in RS-SA sequentially samples the input point clouds. The ARFC module aims to adaptively complement low-level features to account for the information loss caused by the random sampling and inherent non-uniformity. The AR3 module focuses on reasoning about relationships among high-level regional features. The semantic and spatial aware discriminative loss is proposed to supervise the instance embedding learning leveraging the semantic and spatial knowledge. • Random sampling is introduced to improve set abstraction efficiency. • A region ensemble structure is developed to enhance point cloud comprehension. • A semantic spatial awareness is proposed to mitigate instance confusion problem. • Experimental results demonstrate that the proposed method achieves promising boosts. The semantic instance segmentation task on 3D data has made great progress. However, for unstructured 3D point cloud data, the mining of regional knowledge and explicit assistance of semantic for the instance segmentation task are still rarely explored. In this paper, we propose a region ensemble structure including random sampling-based set abstraction (RS-SA), an adaptive regional feature complementary (ARFC) module, and an affinity-based regional relational reasoning (AR3) module in feature encoding to enhance the point cloud comprehension. Random sampling is introduced into the set abstraction of feature encoding to improve computational efficiency. The ARFC module aims to complement low-level features to adaptively compensate for the information loss caused by random sampling and inherent non-uniformity of point clouds, and the AR3 module emphasizes mining the potential reasoning relationships among high-level features based on affinity. Furthermore, a novel semantic spatial aware discriminative loss is proposed to improve the discrimination of instance embedding. The proposed region ensemble structure and semantic spatial awareness for discriminative loss are demonstrated promising boosts on the 3D point cloud semantic instance segmentation task, and the framework achieves state-of-the-art performance on the S3DIS, ScanNet-v2, and ShapeNet datasets. [ABSTRACT FROM AUTHOR]

Machine learning based decision making systems in safety critical areas place high demands on the accuracy and generalization ability of the underlying model. A common strategy to deal with uncertainties and possible mistakes is offered by learning with reject option, i.e. a model can refrain from prediction in ambiguous cases and leave the decision to a human expert. Yet, as for the models themselves, human decision-making is hampered by the fact that reject options are often implemented as black-box rules: Experts cannot readily understand the reasons for rejection. In this work, we propose a model-agnostic framework that enriches classification with reject option by explanation mechanisms. More specifically, we combine conformal prediction as a popular mathematically based technology of certainty estimation with local surrogates derived for the region of interest. This allows us to provide local explanations in terms of example-based explanation methods, including counterfactual, semi-factual, and factual methods. We demonstrate the performance of this technology through a series of benchmarks using 6 different data sets; the associated code is open source. 1 1 This article represents a substantially extended version of the conference paper Artelt et al. (2022). In addition to a more detailed explanation and derivation of the algorithms, we explore semi-factual and factual explanation methods and provide evaluations on realistic benchmark data sets. • New challenge for XAI: explain model uncertainty and reject option • Efficient realization based on counterfactual explanation of local surrogate. • Open source implementation and evaluation on benchmarks. [ABSTRACT FROM AUTHOR]

In recent years, cross-modal retrieval technology has attracted extensive attention with the massive growth of multimedia data. However, most cross-modal hashing methods mainly focus on exploring the retrieval of seen classes, while ignoring the retrieval of unseen classes. Therefore, traditional cross-modal hashing methods cannot achieve satisfactory performances in zero-shot retrieval. To mitigate this challenge, in this paper, we propose a novel zero-shot cross-modal retrieval method called discrete asymmetric zero-shot hashing(DAZSH), which fully exploits the supervised knowledge of multimodal data. Specifically, it integrates pairwise similarity, class attributes and semantic labels to guide zero-shot hashing learning. Moreover, our proposed DAZSH method combines the data features with the class attributes to obtain a semantic category representation for each category. Therefore, the relationships between seen and unseen classes can be effectively captured by learning a category representation vector for each instance. Therefore, the supervised knowledge can be transferred from the seen classes to the unseen classes. In addition, we develop an efficient discrete optimization strategy to solve the proposed model. Massive experiments on three benchmark datasets show that our proposed approach has achieved promising results in cross-modal retrieval tasks. The source code of this paper can be obtained from https://github.com/szq0816/DAZSH. [ABSTRACT FROM AUTHOR]

This paper designs novel command filters (CFs) and a singularity-free controller with fixed-time performance for strict feedback nonlinear systems with input dead-zone. Compared with traditional filter designs, such as dynamic surface control and CFs, the novel CFs designed in this paper can guarantee fixed-time convergence of filter errors and compensating signals. Moreover, the controller design excludes the singularities of control signals, which might result in severe oscillation and usually appear in the conventional fixed-time controller design. To compensate the input dead-zone, a compensation filter is constructed, where controller design and stability analysis can be simplified. In the end, a simulation example is given to verify the effectiveness of the theoretical results. [ABSTRACT FROM AUTHOR]

• The paper fills the gap in the literature by reviewing and analyzing the current studies that apply deep learning for Covid-19 forecasting. • The initial search identified 152 studies of which 53 passed the quality control. • The existing literature is categorized using a model-based taxonomy. • The description of the models along with their performance evaluation is presented. • Recommendations for future improvements are provided. The Covid-19 pandemic has galvanized scientists to apply machine learning methods to help combat the crisis. Despite the significant amount of research there exists no comprehensive survey devoted specifically to examining deep learning methods for Covid-19 forecasting. In this paper, we fill the gap in the literature by reviewing and analyzing the current studies that use deep learning for Covid-19 forecasting. In our review, all published papers and preprints, discoverable through Google Scholar, for the period from Apr 1, 2020 to Feb 20, 2022 which describe deep learning approaches to forecasting Covid-19 were considered. Our search identified 152 studies, of which 53 passed the initial quality screening and were included in our survey. We propose a model-based taxonomy to categorize the literature. We describe each model and highlight its performance. Finally, the deficiencies of the existing approaches are identified and the necessary improvements for future research are elucidated. The study provides a gateway for researchers who are interested in forecasting Covid-19 using deep learning. [ABSTRACT FROM AUTHOR]

Due to the rapid development of electronic journals, selecting appropriate journals to publish research papers has become a significant challenge to researchers. Sometimes, even a high-quality paper may get rejected from the editor due to the mismatch between the topic of the paper and the scope of the journal. To address this issue, we present a framework of learning to recommend journals for submission based on embedding models to assist researchers in journal selection. Specifically, the journal recommendation problem is formulated in the context of multi-class classification, where the Bidirectional Encoder Representations from Transformers (BERT) is deployed to extract the text-level features of representing papers and the AutoEncoder (AE) network is adopted to obtain the feature representation of each journal from the relationship matrix of the paper-journal bipartite graph. The final recommendation of journals is made by using a scoring function and a Softmax classifier. Experimental results obtained on the closed dataset of 10 different journals and the DBLP dataset indicate that we proposed method outperforms several classical approaches in terms of accuracy, F1, MRR, etc. Furthermore, we introduce information entropy as an evaluation index and analyze the model performance from the perspective of prediction uncertainty. This study provides a new approach to the journal recommendation task, and researchers can choose the appropriate embedding methods according to the actual problem. [ABSTRACT FROM AUTHOR]

In this paper, we study the performance of robust learning with Huber loss. As an alternative to traditional empirical risk minimization schemes, Huber regression has been extensively used in machine learning. A new comparison theorem is established in the paper, which characterizes the gap between the excess generalization error and the prediction error. In addition, we refine the error bounds from the perspective of statistical learning theory and improve the convergence rates in the presence of heavy-tailed noise. It is worth mentioning that a new moment condition E [ | Y | 1 + ∊ | X = x ] ∈ L ρ X 2 is employed in analysis of error bound and learning rates from a theoretical viewpoint. [ABSTRACT FROM AUTHOR]

Video summarization (VS) technology presents a multi-tendency development. However, there are still certain challenges in shortening a long video into several diverse and concise versions, such as generating various versions and variable lengths of summaries depending on user requirements. In this context, it is critical to extract the spatial–temporal features of video frames. To address these issues, this paper proposes a multi-flexible video summarization scheme using a property-constraint decision. Due to the difficulty of directly transforming user requirements into summary algorithm factors, this paper employs property-constraints as a bridge between user requirements and the video summary algorithm. The specific implementation method is to aggregate and analyze existing research results on summary-oriented properties. It begins with the establishment of a property-constraints library, followed by the translating and mapping of user requirements into VS property-constraints, and ultimately constructing a property-constraints tree for flexible decision versions of VS. In addition, a hybrid cascade bidirectional network (CB-ConvLSTM) based on the Convolutional LSTM Network (ConvLSTM) is designed to extract the spatial–temporal features of the video. On this basis, the VS generator is configured. The goal of this scheme, which combines the VS property constraints and CB-ConvLSTM, is to "analyze once, satisfy multiple factors, generate multiple levels". Verification experiments and a comparative analysis are conducted on benchmark datasets to evaluate the algorithm in this paper. The results indicate that the proposed algorithm is highly rational, effective, and usable. [ABSTRACT FROM AUTHOR]

• An overview of deep learning algorithms used in medical image segmentation is presented. • More than 150 papers applying deep learning to different medical applications are summarised. • Challenges and future directions in medical image segmentation are discussed. Deep learning (DL) algorithms have rapidly become a robust tool for analyzing medical images. They have been used extensively for medical image segmentation as the first and significant components of the diagnosis and treatment pipeline. Medical image segmentation is efficiently addressed by many types of deep neural networks, such as convolutional neural networks, fully convolutional network recurrent networks, adversarial networks, and U-shaped networks. This paper reviews the major DL models and applications pertinent to medical image segmentation and summarizes over 150 contributions to the field. Brief overviews of articles are provided by application area: anatomical structures such as organs, bones, and vessels, and abnormalities such as lesions and calcification. Moreover, we discuss current challenges and suggest directions for future research. [ABSTRACT FROM AUTHOR]

Time interval between stimuli plays a critical role in Pavlov classical conditioning. In this paper, a full-functional memristive Pavlov associative memory circuit with inter-stimulus interval effect is proposed. Compared with previous works, the proposed circuit extends the Pavlov associative memory with time interval to three complex situations of conditioning and realizes bionic associative memory learning curves called inter-stimulus interval effect. On this basis, the phenomena related to time interval in classical conditioning, blocking and facilitation, are implemented. PSPICE is used to simulate the whole circuit, and the simulation results demonstrate above functions. Furthermore, an application on classification is extended on the basis of the proposed circuit. The application circuit is also verified by the simulation results in PSPICE. This paper provides a reference to associative memory based on memristor in complex time situations. [ABSTRACT FROM AUTHOR]

• A detailed literature review for the existing methods about anchor assignment and sample sampling is investigated and analyzed in a taxonomy. We identify a definition of each component, challenging problems in existing methods and their solution, and a comprehensive comparison of all methods. To the best of our knowledge, there is no prior work discussing these problems in object detection literature. • Open research direction in both anchor assignment and sampling heuristics is discussed. Based on these directions, we hope the object detection community can identify the current status of object detection methods to propose better solutions to anchor assignment and sampling heuristics. • We review the state-of-the-art object detection in the last two years that complement existing surveys. We also provide paper literature related to the above problems via the repository webpage. Deep learning-based object detection is a fundamental but challenging problem in computer vision field, has attracted a lot of study in recent years. State-of-the-art object detection methods rely on the selection of positive samples and negative samples, i.e., called sample assignment, and the definition of a useful set for training, i.e., called sample sampling heuristics. This paper presents a comprehensive review of the advanced anchor assignment and sampling approaches in deep learning-based object detection. Each problem is classified and analyzed systematically. According to the problem-based taxonomy, we identify the advantages and disadvantages of each problem in-depth and present open issues regarding the current methods. Furthermore, this paper also reviews the new trends in solving object detection that has not been discussed during the last two years. To track the latest research, a webpage related to the above problems is provided, which is available at https://github.com/VoXuanThuy/ObjectDetectionReview. [ABSTRACT FROM AUTHOR]

Learning hidden topics from short text streams has become crucial in modern applications such as social networks, instant messages, question and answer forums, etc. Building an effective learning method poses two main challenges: Short and noisy data, as well as the stability-plasticity dilemma. In this paper, we investigate carefully how existing methods face these challenges. From our theoretical and empirical analyses, they often deal well with a challenge but ineffectively handle the other. In particular, they often suffer from catastrophic forgetting, because they impose a constraint on the learned knowledge from only the previous time step in the streaming duration. In this paper, we propose a novel method, namely BSP, which has a regularization term based on second-order Taylor expansion to accumulate information from all former minibatches. Moreover, external knowledge and Dropout technique will be combined at each time step to handle better short and noisy texts as well as enhance the model's plasticity. We empirically evaluate BSP, compared to other state-of-the-art streaming methods in terms of dealing with stability-plasticity dilemma and handling short and noisy texts. The extensive experiments show superior effectiveness of BSP. [ABSTRACT FROM AUTHOR]

• Model-based and model-free gait recognition techniques apt for handling covariates. • An exploration of benefits/ limitations of deep learning pipelines under covariates. • List of covariates-based gait datasets for training robust gait recognition system. • Overview of the proposed fusion-based pipeline with the 3D dataset for covariates. • Research gaps and challenges in the implementation of deep learning pipelines. Gait recognition is an essential biometric technique that recognizes humans at a distance through their unique walking style. In the present era of deep learning, automated gait analysis has become easier with an increase in processing power. However, the recognition accuracy is affected by many covariates such as clothing conditions, carrying objects, varying viewing angles, occlusion, walking speed variations, and thus, it remains a challenging problem. For this complex problem, huge datasets are required to train for given conditions and predict new situations; thus, deep learning is preferred. In this review paper, we categorize various gait covariates which have been recently handled. There are various approaches, but the most effective approach is deep learning; hence in this paper, we include the most used deep learning approaches for each covariate condition found in the literature. Further, we highlight open problem areas handling these covariates and offer some suggestions about their better handling. Based on the review and our understanding of all the gait pipelines employed in deep learning, we have suggested a comprehensive and universal deep learning pipeline that can handle most gait covariates rather than customized deep learning pipelines. The methods of handling gait covariates are summarized according to appearance, pose, and sensors. A comprehensive comparison of reviewed approaches for real-time scenarios in terms of their novelty, benefits, and limitations is then offered, which led us to identify open research problems related to gait covariates. In the end, the paper concludes with the challenges identified and prospects. [ABSTRACT FROM AUTHOR]

Foreign species can deteriorate the environment and the economy of a country. To automatically monitor biosecurity threats at country borders, this paper investigates compact deep networks for accurate and real-time object segmentation for hyperspectral images. To this end, knowledge distillation (KD) approaches compress the model by distilling the knowledge of a large teacher network to a compact student network. However, when the student is over-compressed, the performance of standard KD methods degrades significantly due to the large capacity gap between the teacher and the student. This gap can be addressed by adding medium-sized teacher assistants, but training them incurs significant computation and hence is impractical. To address this problem, this paper proposes a new framework called Knowledge Distillation from Multi-head Teacher (KDM), which distills the knowledge of a multi-head teacher to the student. By encapsulating multiple teachers in a single network, our proposed KDM assists the learning of a very compact student and significantly reduces the training time. We also introduce Bio-HSI, a new large benchmark hyperspectral image dataset of 3,125 high-resolution images with dense segmentation ground truth. This new, large dataset can be expected to advance research on deep models for hyperspectral image segmentation. Evaluated on this dataset, the student trained via our KDM has 762 times fewer parameters than the state-of-the-art segmentation model (i.e., HRNet), while achieving competitive accuracy. [ABSTRACT FROM AUTHOR]

Computational research on facial micro-expressions has long focused on videos captured under constrained laboratory conditions due to the challenging elicitation process and limited samples that are publicly available. Moreover, processing micro-expressions is extremely challenging under unconstrained scenarios. This paper introduces, for the first time, a completely automatic micro-expression "spot-and-recognize" framework that is performed on in-the-wild videos, such as in poker games and political interviews. The proposed method first spots the apex frame from a video by handling head movements and unconscious actions which are typically larger in motion intensity, with alignment employed to enforce a canonical face pose. Optical flow guided features play a central role in our method: they can robustly identify the location of the apex frame, and are used to learn a shallow neural network model for emotion classification. Experimental results demonstrate the feasibility of the proposed methodology, establishing good baselines for both spotting and recognition tasks – ASR of 0.33 and F1-score of 0.6758 respectively on the MEVIEW micro-expression database. In addition, we present comprehensive qualitative and quantitative analyses to further show the effectiveness of the proposed framework, with new suggestion for an appropriate evaluation protocol. In a nutshell, this paper provides a new benchmark for apex spotting and emotion recognition in an in-the-wild setting. [ABSTRACT FROM AUTHOR]

• FLA has attracted attention of researchers for its research significance and application. • This paper attempts to give an overview of fitness landscape analysis for optimization. • Future research works are discussed and given in four aspects. Over past few decades, as a powerful analytical tool to characterize the fitness landscape of a problem, fitness landscape analysis (FLA) has been widely concerned and utilized for all kinds of optimization areas. Since its introduction by Sewell Wright in 1932, FLA has attracted more and more attention of researchers for its research significance and application value such as an intuitive understanding features of complex optimization problems, explaining evolutionary algorithm behavior, assessing performances of algorithms, and guiding selections and/or configures of algorithms. This paper attempts to give an overview of fitness landscape analysis and its typical application for optimization so far. We hope that this survey can help to understand features of complex optimization problems in depth and thus to improve the certain algorithm performance of for a given optimization problem. [ABSTRACT FROM AUTHOR]

Blending free annotation cost, low memory usage and high query speed into a unity, unsupervised deep hashing has shown extraordinary talents in image retrieval. In order to address the lack of semantic information in unsupervised scenario, related works leverage models pretrained on large-scale datasets (e.g., ImageNet) to the estimate semantic similarity for specific datasets (e.g., FLICKR-25 K). However, most of them discard the original semantic attribute in the pretrained datasets, which results in hash codes being over-fitted to the estimated semantic similarity information. To alleviate this problem, this paper tries to consider the semantic similarity of the specific dataset and preserve the semantic attribute from the pretrained dataset simultaneously. Meanwhile, to build a reliable semantic similarity matrix for the specific dataset, this paper develops a similarity distiller by jointly measuring the semantic similarity with the distance between deep features, the distance on the local manifold and the distance on the connectivity graph. Moreover, an efficient attribute preserver is also designed to maintain the correspondence between the hash codes and the category attributes from the pretrained dataset based on regeneration criteria. The method proposed in this paper is named PSIDP for simplicity and extensive image retrieval experiments on four benchmark datasets demonstrate the superiority of the proposed PSIDP method when compared with other state-of-the-art unsupervised deep hashing methods. The code is available at https://github.com/reresearcher/PSIDP. [ABSTRACT FROM AUTHOR]