Search

The complex and varied urban road conditions have always been a difficult and a pivotal component in the study of driverless technology especially at intersections. In China, it is necessary for driverless cars to understand the gestures of traffic police. To identify the traffic police gesture at the intersection, the key step is to detect the traffic police at the intersection. At present, the research on traffic police detection is still in its infancy, there exists common problems such as slow detection speed and other real time problems in this method, and there is not a standardized traffic police data set ether. For the real time problems, this paper introduces the attention mechanism, and proposes a new real-time detection method of traffic police based on attention mechanism. The method proposed in this paper has strong robustness and can quickly complete the target detection task. For the data set problem, this paper analyzes and discloses similar data sets published through research. In the meanwhile, this paper collects 24,530 video data on the actual road, and it extracts and saves 12,000 pictures containing traffic police at frame rate of 30FPS as traffic police detection training and validated data set. [ABSTRACT FROM AUTHOR]

Quite often a machine learning problem lends itself to be split in several well-defined subproblems, or tasks. The goal of Multi-Task Learning (MTL) is to leverage the joint learning of the problem from two different perspectives: on the one hand, a single, overall model, and on the other hand task-specific models. In this way, the found solution by MTL may be better than those of either the common or the task-specific models. Starting with the work of Evgeniou et al., support vector machines (SVMs) have lent themselves naturally to this approach. This paper proposes a convex formulation of MTL for the L1-, L2- and LS-SVM models that results in dual problems quite similar to the single-task ones, but with multi-task kernels; in turn, this makes possible to train the convex MTL models using standard solvers. As an alternative approach, the direct optimal combination of the already trained common and task-specific models can also be considered. In this paper, a procedure to compute the optimal combining parameter with respect to four different error functions is derived. As shown experimentally, the proposed convex MTL approach performs generally better than the alternative optimal convex combination, and both of them are better than the straight use of either common or task-specific models. [ABSTRACT FROM AUTHOR]

• Analytical developments support the understanding of ELM variability. • A special attention is paid to the impact of the input weight randomness. • Variance estimates for homoskedastic, heteroskedastic, regularized cases are given. • The possibility of constructing accurate confidence intervals is discussed. • A new Python library allows the computation of the proposed estimates. Uncertainty quantification is crucial to assess prediction quality of a machine learning model. In the case of Extreme Learning Machines (ELM), most methods proposed in the literature make strong assumptions on the data, ignore the randomness of input weights or neglect the bias contribution in confidence interval estimations. This paper presents novel estimations that overcome these constraints and improve the understanding of ELM variability. Analytical derivations are provided under general assumptions, supporting the identification and the interpretation of the contribution of different variability sources. Under both homoskedasticity and heteroskedasticity, several variance estimates are proposed, investigated, and numerically tested, showing their effectiveness in replicating the expected variance behaviours. Finally, the feasibility of confidence intervals estimation is discussed by adopting a critical approach, hence raising the awareness of ELM users concerning some of their pitfalls. The paper is accompanied with a scikit-learn compatible Python library enabling efficient computation of all estimates discussed herein. [ABSTRACT FROM AUTHOR]

Intrusion Detection, the ability to detect malware and other attacks, is a crucial aspect to ensure cybersecurity. So is the ability to identify this myriad of attacks. Artificial Neural Networks (as well as other machine learning bio-inspired approaches) are an established and proven method of accurate classification. ANNs are extremely versatile – a wide range of setups can achieve significantly different classification results. The main objective and contribution of this paper is the evaluation of the way the hyperparameters can influence the final classification result. In this paper, a wide range of ANN setups is put to comparison. We have performed our experiments on two benchmark datasets, namely NSL-KDD and CICIDS2017. The most effective arrangement achieves the multi-class classification accuracy of 99.909% on an established benchmark dataset. [ABSTRACT FROM AUTHOR]

• ANN, CART, and RF regression trees have shown the potential in modeling energy cost. • Three different strategies regarding variable selection were tested and compared. • Machine learning and Boruta method have produced the highest accuracy of prediction. • The model has extracted heating and occupational data as the most important. • The created model could be used to assess the concept of smart buildings and cities. The paper deals with modeling the cost of energy consumed in public buildings by leveraging three machine learning methods: artificial neural networks, CART, and random forest regression trees. Energy consumption is one of the major issues in global and national policies, therefore scientific efforts in creating prediction models of energy consumption and cost are highly important. One of the largest energy consumers in every state is its public sector, consisting of educational, health, public administration, military, and other types of public buildings. Recent technologies based on sensor networks and Big data platforms enable collection of large amounts of data that could be used to analyze energy consumption and cost. A real data from Croatian public sector is used in this paper including a large number of constructional, energetic, occupational, climate and other attributes. The algorithms for data pre-processing and modeling by optimizing parameters are suggested. Three strategies were tested: (1) with all available variables, (2) with a filter-based variable selection, and (3) with a wrapper-based variable selection which integrates Boruta algorithm and random forest. Prediction models of energy cost are created using two approaches: (a) comparative usage of artificial neural networks and two types of regression trees, CART and random forest, and (b) integration of RF-Boruta variable selection and machine learning methods for prediction. A cross-validation procedure was used to optimize the artificial neural network and regression tree topology, as well to select the most appropriate activation function. Along with creating a prediction model, the aim of the paper was also to extract the relevant predictors of energy cost in public buildings which are important in planning the construction or renovation of buildings. The results have shown that the second approach which integrates machine learning with Boruta method, where the random forest algorithm is used for both variable reduction and prediction modeling, has produced a higher accuracy of prediction than the individual usage of three machine learning methods. Such findings confirm the potential of hybrid machine learning methods which are suggested in previous research, but in favor of random forest method over CART and artificial neural networks. Regarding the variable selection, the model has extracted heating and occupational data as the most important, followed by constructional, cooling, electricity, and lighting attributes. The model could be implemented in public buildings information systems and their IoT networks within the concept of smart buildings and smart cities. [ABSTRACT FROM AUTHOR]

[Display omitted] • We propose a novel distillation approach, i.e., LGAD, which uses structure hints in lane labels to guide the attention of lane segmentation networks to better capture long-range textural information. • We carefully investigate the inner mechanism of LGAD, the considerations of choosing among different layer mimicking paths, and the optimization strategies. • Extensive experiments on two benchmark datasets demonstrate the effectiveness of the proposed LGAD on boosting the performance of lane segmentation networks. Contemporary segmentation methods are usually based on deep fully convolutional networks (FCNs). However, the layer-by-layer convolutions with a growing receptive field is not good at capturing long-range contexts such as lane markers in the scene. In this paper, we address this issue by designing a distillation method that exploits label structure when training segmentation network. The intuition is that the ground-truth lane annotations themselves exhibit internal structure. We broadcast the structure hints throughout a teacher network, i.e., we train a teacher network that consumes a lane label map as input and attempts to replicate it as output. Then, the attention maps of the teacher network are adopted as supervisors of the student segmentation network. The teacher network, with label structure information embedded, knows distinctly where the convolutional layers should pay visual attention into. The proposed method is named as Label-guided Attention Distillation (LGAD). It turns out that the student network learns significantly better with LGAD than when learning alone. As the teacher network is deprecated after training, our method does not increase the inference time. Note that LGAD can be easily incorporated in any lane segmentation network. To validate the effectiveness of the proposed LGAD method, extensive experiments have been conducted on two popular lane detection benchmarks: TuSimple and CULane. The results show consistent improvement across a variety of convolutional neural network architectures. Specifically, we demonstrate the accuracy boost of LGAD on the lightweight model ENet. It turns out that the ENet-LGAD surpasses existing lane segmentation algorithms. The main contributions of this paper include a newly proposed distillation training strategy (LGAD) and solid experimental investigation of the inner mechanism of LGAD. [ABSTRACT FROM AUTHOR]

Road landmark inventory is becoming an important data product for the maintenance of transport infrastructures. Several commercial sensors are available which include synchronized optical cameras that allowto build 360° panoramic images of the surroundings of the vehicle used for road inspection. This paper is devoted to the analysis of such panorama images,specifically the area that contains themost relevant information. Road lane landmark detection is posed as a two class classification problem that may be solved bymachine learningapproaches, such as Random Forest (RF) and ensembles of Extreme Learning Machines (V-ELM). Besides model parameter selection, a central problem is the construction of a labeled training and validation datasetto cope with the highly uncontrolled conditions of image capture. Besides, human labor cost makes image data labeling a very expensive process. This paper proposes an open ended Active Learning (AL) approach involving a human oraclein the loop who provides the data labeling and can trigger the AL process when detection quality is degraded by the change in imaging conditions. The paper reports encouraging results over a collection of sample images selected from an industrial road landmark inventory operation. As an additional contribution, this paper assesses the ability of AL to overcomesome of the issues raised by highly class imbalanced datasets. [ABSTRACT FROM AUTHOR]

With the advent of smart phones, the popularity of free Android applications has risen rapidly. This has led to malicious Android apps being involuntarily installed, which violate the user privacy or conduct attack. Malware detection on Android platforms therefore is a growing concern because of the undesirable similarity between malicious behavior and benign behavior, which can lead to slow detection, and allow compromises to persist for comparatively long periods of time in infected phones. The contributions of this paper are first a multiple dimensional, kernel feature-based framework and feature weight-based detection (WBD) designed to categorize and comprehend the characteristics of Android malware and benign apps. Furthermore, our software agent is orchestrated and implemented for the data collection and storage to scan thousands of benign and malicious apps automatically. We examine 112 kernel attributes of executing the task data structure in the Android system and evaluate the detection accuracy with a number of datasets of various dimensions. We find that memory- and signal-related features contribute to more precise classification than schedule-related and other descriptors of task states listed in our paper. Particularly, memory-related features provide fine-grain classification policies for preserving higher classification precision than the signal-related and others. Furthermore, we study and evaluate 80 newly infected attributes of the Android kernel task structure, prioritizing the 70 features of most significance based on dimensional reduction to optimize the efficiency of high-dimensional classification. Our second contribution is that our experiments demonstrate that, as compared to existing techniques with a short list of task structure features (16 or 32 features), our method can achieve 94%-98% accuracy and 2%–7% false positive rate, while detecting malware apps with reduced-dimensional features that adequately abbreviate online malware detections and advance offline malware inspections. [ABSTRACT FROM AUTHOR]

There remains an open question about the usefulness and the interpretation of machine learning (ML) approaches for discrimination of spatial patterns of brain images between samples or activation states. In the last few decades, these approaches have limited their operation to feature extraction and linear classification tasks for between-group inference. In this context, statistical inference is assessed by randomly permuting image labels or by the use of random effect models that consider between-subject variability. These multivariate ML-based statistical pipelines, whilst potentially more effective for detecting activations than hypotheses-driven methods, have lost their mathematical elegance, ease of interpretation, and spatial localization of the ubiquitous General linear Model (GLM). Recently, the estimation of the conventional GLM parameters has been demonstrated to be connected to an univariate classification task when the design matrix in the GLM is expressed as a binary indicator matrix. In this paper we explore the complete connection between the univariate GLM and ML-based regressions. To this purpose we derive a refined statistical test with the GLM based on the parameters obtained by a linear Support Vector Regression (SVR) in the inverse problem (SVR-iGLM). Subsequently, random field theory (RFT) is employed for assessing statistical significance following a conventional GLM benchmark. Experimental results demonstrate how parameter estimations derived from each model (mainly GLM and SVR) result in different experimental design estimates that are significantly related to the predefined functional task. Moreover, using real data from a multisite initiative the proposed ML-based inference demonstrates statistical power and the control of false positives, outperforming the regular GLM. [ABSTRACT FROM AUTHOR]

• The proposed method introduces objectness constraint by using multiple output heads. • The target position consistency is enhanced by introducing positional encoding. • The multi-head soft aggregation based instance attention is designed. • Our method surpasses previous top performers and maintains a competitive speed. Objectness is a notion of how a generic object looks like. Recently, matching-based methods have yielded impressive video object segmentation results. However, these methods suffer from mis-segmentation of distractors that are distant from the targets due to the lack of objectness constraint. The segmentation mask has a very low objectness when both the correct object and the distant distractor are considered as a target. Position information is also ignored in matching-based methods. In this paper, a video object segmentation method based on target objectness constraint and multi-head soft aggregation is proposed. The target objectness constraint is introduced by forcing the network to learn to predict object center and pixel-to-object boundary distances. The problem of mis-segmentation of objects or regions with similar appearance at a distance is alleviated. Besides, the target position consistency constraint is strengthened by introducing the target position information in the feature matching of adjacent frames. Furthermore, an instance attention mechanism is implemented using multi-head soft aggregation to re-calibrate channel importance. Multi-head soft aggregation enables better utilization of target feature information than global average pooling. Comprehensive experiments demonstrate that the proposed method obtains a competitive performance, surpassing many other existing methods. [ABSTRACT FROM AUTHOR]

Ensemble learning has gained success in machine learning with major advantages over other learning methods. Bagging is a prominent ensemble learning method that creates subgroups of data, known as bags, that are trained by individual machine learning methods such as decision trees. Random forest is a prominent example of bagging with additional features in the learning process. Evolutionary algorithms have been prominent for optimisation problems and also been used for machine learning. Evolutionary algorithms are gradient-free methods that work with a population of candidate solutions that maintain diversity for creating new solutions. In conventional bagged ensemble learning, the bags are created once and the content, in terms of the training examples, are fixed over the learning process. In our paper, we propose evolutionary bagged ensemble learning, where we utilise evolutionary algorithms to evolve the content of the bags in order to iteratively enhance the ensemble by providing diversity in the bags. The results show that our evolutionary ensemble bagging method outperforms conventional ensemble methods (bagging and random forests) for several benchmark datasets under certain constraints. We find that evolutionary bagging can inherently sustain a diverse set of bags without reduction in performance accuracy. [ABSTRACT FROM AUTHOR]

Most text classification methods achieve great success based on the large-scale annotated data and the pre-trained language models. However, the labeled data is insufficient in practice, and the pre-trained language models are difficult to deploy due to their high computing resources and slow inference speed. In this paper, we propose cross-domain knowledge distillation, where the teacher and student tasks belong to different domains. It not only acquires knowledge from multiple teachers but also accelerates inference and reduces model size. Specifically, we train the pre-trained language models on factual knowledge obtained by aligned Wikipedia text to Wikidata triplets and fine-tune it as the teacher model. Then we use the heterogeneous multi-teacher knowledge distillation to transfer knowledge from the multiple teacher models to the student model. Multi-teacher knowledge vote can distill knowledge related to the target domain. Moreover, we also introduce the teacher assistant to help distill large pre-trained language models. Finally, we reduce the difference between the source domain and target domain by multi-source domain adaptation to solve the domain shift problem. Experiments on the multiple public datasets demonstrate that our method can achieve competitive performance while having fewer parameters and less inference time. [ABSTRACT FROM AUTHOR]

It has been widely noticed the performance of algorithms for high-resolution face recognition (HRFR) degrades significantly for low-resolution face recognition (LRFR). In this paper, we discover the main source of this performance degradation comes from the human-defined inductive bias of CNN, which constrains the model to absorb effective information and leads to overfitting. To overcome the shortcoming, for the first time, we adopt a transformer-based DNN algorithm named DeiT to accomplish LRFR tasks. On the other hand, we further borrow the form of knowledge distillation. Traditional knowledge distillation network for LRFR sets the student model to be simpler than or the same as the teacher model while using the teacher model off-the-shelf, leading to the model capacity gap. Instead, we fuse an online network into the original parameter-fixed teacher model to learn how to transfer knowledge. The final "knowledge" is the sum fusion of the outputs of both the teacher model and the student model. Experiments show that even without training on LR face images, the performance of our model can be comparable to recent baselines delicately designed for LRFR tasks on certain LR face datasets. After the entire training process, in either real-world LR datasets, artificially down-sampled datasets, or generated LR face datasets, our method performs favorably against the state-of-the-art (SOTA) algorithms. [ABSTRACT FROM AUTHOR]

Recently, the attention mechanism became the key issue for image super-resolution (SR) because it has the ability to extract different features from the image according to the used attention type. Despite the great success of attention-based methods, the conflict among features of different attention types can affect the SR performance. In this paper, we propose an efficient single image SR model called balanced spatial feature distillation and pyramid attention (BSPAN). The idea of BSPAN is based on the trade-off among the extracted features of different attention types. Also, we propose a balanced spatial feature distillation block (BSFDB) as the backbone of BSPAN so that the network can effectively advantage from the different attention features. Two different attention types, namely spatial attention residual feature distillation (SARFD) and classical attention (CA) are considered in the BSFDB to achieve a leveling between them based on the contents of the low-resolution feature map using the balancing attention block. The BSFDB block is designed to improve the SR performance and has lightweight parameters and low computation complexity. Moreover, to further improve the SR performance, the pyramid attention is introduced in the middle of the BSPAN network for extracting long-range features at a variety of locations and scales. Evaluation based on five benchmark datasets, we concluded that the balancing between features of a variety of attention types can effectively improve the SR performance. So, the proposed BSPAN model achieves significant enhancements in comparison with the state-of-the-art and superior visual quality and reconstruction accuracy. [ABSTRACT FROM AUTHOR]

• We study the problem of distributed image-to-image translation. • A novel framework for distributed image-to-image translation. • A unified framework for distributed image-to-image translation and autoencoding. Data-driven paradigms using machine learning are becoming ubiquitous in image processing and communications. In particular, image-to-image (I2I) translation is a generic and widely used approach to image processing problems, such as image synthesis, style transfer, and image restoration. At the same time, neural image compression has emerged as a data-driven alternative to traditional coding approaches in visual communications. In this paper, we study the combination of these two paradigms into a joint I2I compression and translation framework, focusing on multi-domain image synthesis. We first propose distributed I2I translation by integrating quantization and entropy coding into an I2I translation framework (i.e. I2Icodec). In practice, the image compression functionality (i.e. autoencoding) is also desirable, requiring to deploy alongside I2Icodec a regular image codec. Thus, we further propose a unified framework that allows both translation and autoencoding capabilities in a single codec. Adaptive residual blocks conditioned on the translation/compression mode provide flexible adaptation to the desired functionality. The experiments show promising results in both I2I translation and image compression using a single model. [ABSTRACT FROM AUTHOR]

In this paper, the measured input–output data sequences with pinning gain are proposed via the topology of multi-agent systems (MAS), where the requirement of reinforcement learning algorithm on internal state is avoided by pinning gain and the measured data of leader and neighbors. Besides, a data-based tracking state is given, which can be applied to MAS with different control matrices. According to the sequences and tracking state, a distributed control policy and corresponding reinforcement learning algorithm are proposed for the output synchronization. The proposed algorithm overcomes the shortcoming that previous algorithms can not be applied to MAS with different control matrices in the absence of model information and full-state vector. Finally, the effectiveness of proposed algorithm is verified by simulation examples. [ABSTRACT FROM AUTHOR]

For NP-hard combinatorial optimization problems, it is usually challenging to find high-quality solutions in polynomial time. Designing either an exact algorithm or an approximate algorithm for these problems often requires significantly specialized knowledge. Recently, deep learning methods have provided new directions to solve such problems. In this paper, an end-to-end deep reinforcement learning framework is proposed to solve this type of combinatorial optimization problems. This framework can be applied to different problems with only slight changes of input, masks, and decoder context vectors. The proposed framework aims to improve the models in literacy in terms of the neural network model and the training algorithm. The solution quality of TSP and the CVRP up to 100 nodes are significantly improved via our framework. Compared with the best results of the state-of-the-art methods, the average optimality gap is reduced from 4.53% to 3.67% for TSP with 100 nodes and from 7.34% to 6.68% for CVRP with 100 nodes when using the greedy decoding strategy. Besides, the proposed framework can be used to solve a multi-depot CVRP case without any structural modification. Furthermore, our framework uses about 1/3 ∼ 3/4 training samples compared with other existing learning methods while achieving better results. The results performed on randomly generated instances, and the benchmark instances from TSPLIB and CVRPLIB confirm that our framework has a linear running time on the problem size (number of nodes) during training and testing phases and has a good generalization performance from random instance training to real-world instance testing. [ABSTRACT FROM AUTHOR]

The success of convolutional neural networks (CNNs) has made low-latency inference services on Graphic Processing Units (GPUs) a hot research topic. However, GPUs are hardware processors with high power consumption. To have the least energy consumption while meeting latency Service-Level-Objective (SLO), batching strategy and dynamic voltage frequency scaling (DVFS) are two important solutions. However, existing studies do not coordinate them and regard CNN as a black box, which makes inference services less energy-efficient. In this paper, we propose EALI, an energy-aware layer-level adaptive scheduling framework that is comprised of a power prediction model, a layer combination strategy, and an energy-aware layer-level scheduler. The power prediction model uses classic machine learning techniques to predict fine-grained layer-level power consumption. The layer combination strategy combines multiple layers into optimization units to lower scheduling overhead and complexity. The energy-aware layer-level scheduler adaptively coordinates batching strategy and layer-level DVFS according to workloads to minimize the energy consumption while meeting SLO. Our experimental results on NVIDIA Tesla M40 and V100 GPUs show that, compared to the state-of-the-art approaches, EALI decreases energy consumption by up to 36.24% while meeting SLO. [ABSTRACT FROM AUTHOR]

Retinal optical coherence tomography (OCT) images provide fundamental information regarding the health of the posterior eye (e.g., the retina and choroid). Thus, the development of automatic image analysis methods (e.g., segmentation) is fundamental to provide clinicians and researchers with quantitative data that facilitates decision making. In recent years, various machine learning (ML) methods have been developed to perform these automated image analyses, improving performance over traditional methods, increasing repeatability, and reducing the use of time-consuming manual analysis. Deep learning (DL), a sub-field of ML, represents a new approach that can improve image processing outcomes. Results published to date demonstrate that DL architectures generally achieve superior performance to previously proposed methods based on traditional image analysis or early ML techniques. Thus, DL methods for OCT image analysis have provided an important advance in the area of retinal layer segmentation in images from healthy eyes as well as those with pathologies. This paper provides a comprehensive narrative literature review of current DL layer segmentation methods applied to OCT images of the posterior segment of the eye. The manuscript provides an overview on the state-of-the-art in deep learning as well as highlighting some important areas for future developments to extend the analysis methods in this field. [ABSTRACT FROM AUTHOR]

Previous work like Category Structure Knowledge Distillation proposes to construct category-wise relations for knowledge distillation by introducing intra-category and inter-category relations based on category centers. However, category centers may be unreliable when feature representations of wrongly classified samples are used to form category centers. Besides, inter-category relations based on category centers are coarse-grained. In this paper, we propose a Marginal Sample Knowledge Distillation (MSKD) to construct reliable category centers and fine-grained inter-category relations by introducing label filtering and marginal samples. Label filtering removes feature representations of wrongly classified samples from the calculation of category centers to create unbiased and reliable category centers. Marginal samples are defined as the correctly classified samples close to category boundaries. Marginal samples contain information of other categories and form fine-grained category boundaries for knowledge distillation. Extensive experiments on different datasets and teacher-student architecture settings show that our method has excellent performances when compared with closely related methods. [ABSTRACT FROM AUTHOR]

The classification of imbalanced datasets is one of the main challenges in machine learning techniques. Support vector machine (SVM), which generates a model biased within the majority class, usually has bad performance on the minority class because this class may be considered incorrectly as noises. Moreover, datasets often include noises and outliers, and SVM cannot effectively deal with those datasets. In this paper, to defeat the aforementioned challenges, we propose intuitionistic fuzzy twin support vector machines for imbalanced data (IFTSVM-ID). The proposed method can easily handle imbalanced datasets in the presence of noises and outliers. A reasonable weighting strategy is offered to deal with imbalanced classes, and a margin-based technique is assigned to reduce the impact of noise and outliers. We formulate the linear and non-linear kernel functions to find two non-parallel hyperplanes. One real-world and thirty-two imbalanced datasets are selected to validate the performance of IFTSVM-ID. The Friedman test and the bootstrap technique with 95% confidence interval are applied to quantify the results statistically. The experimental results show that our proposed method has much better performance in comparison with other similar techniques. [ABSTRACT FROM AUTHOR]

Supervised learning has dominated and gradually evolved into a de facto choice for automated semantic segmentation tasks, in spite of the need for large corpora of pixel-level annotated real-world data. Domain adaptation (DA), especially methods based on adversarial machine learning, can lessen labeling costs by securing that knowledge learned from synthetic (source) data with free annotations is transferable to unlabeled real-world data. However, commonly employed approaches in adversarial-based DA merely consider the correlation alignment between the source and target domains while completely neglecting the uncertainty information introduced from the generator. In this paper, we aim to alleviate this issue by introducing a novel entropy-guided adversarial learning framework with the reconstruction error constraint. Firstly, a perceptual-based color space is employed to transform the synthetic source images into the desired new space to approximate the appearances of the real-world images in the target domain. Secondly, an entropy-weighted adversarial framework is designed to enhance the discriminativeness and transferability of the presented model to the target domain via an autoencoder-based discriminator. In addition, a dynamic pseudo-labeling mechanism is introduced to work in conjunction with the curriculum-based self-training strategy to further improve the domain adaptability of the model. Experimental results on two well-known DA benchmarks demonstrate that the proposed method outperforms existing similar approaches in the task of semantic segmentation. [ABSTRACT FROM AUTHOR]

• We enlarge search space by introducing the Cartesian product of operators. • We propose a factorized search to search separately in two subspaces. • We optimize architectural parameters sequentially, avoiding undesirable competition. • We perform diagnostic experiments and explanations, demonstrating our superiority. • Our approach achieves SOTA performance on CIFAR10, ImageNet, and COCO. Neural Architecture Search (NAS) replaces manually designed networks with automatically searched networks and has become a hot topic in machine learning and computer vision. One key factor of NAS, scaling up the search space by adding operators, could help bring about more possibilities for effective architectures. However, existing works require high search costs and are prone to make confused selections caused by competition among similar operators. This paper presents a scalable NAS that utilizes a factorized method, which improves existing art in the following aspects. (1) Our work explores a broader search space. We construct an ample search space through the cartesian product of activation operators and regular operators. (2) Our work experiences a limited computation burden even though searching in a more extensive search space. We factorize a search space into two subspaces and adopt different architectural parameters to control corresponding subspaces. (3) Our work avoids competition among similarly combined operators (e.g. , ReLU & sep-conv-3x3 and ReLU6 & sep-conv-3x3 and PReLU & sep-conv-3x3). That is because our architectural parameters are optimized sequentially. Experimental results show that our approach achieves SOTA on CIFAR10 (2. 38 % test error) and ImageNet (23. 8 % test error). Furthermore, the excellent performance of 35. 8 % (AP) and 55. 7 % ( AP 50 ) on COCO further proves the superiority of our factorized method. [ABSTRACT FROM AUTHOR]

An electroencephalogram (EEG)–based brain–computer interface (BCI) speller is a system that conveys thought to enable communication between humans and computers using brain conduction. This system is useful for patients with severe disabilities and provides a way for them to communicate with computers or with other people. The EEG-based BCI speller system is being studied in various ways, but it is vulnerable to adversarial examples. An adversarial example is an attack sample created by adding a little noise to an original sample in such a way that it appears normal to humans but will be misclassified by the model. Adversarial examples can be useful in situations such as military scenarios in which there is a mixture of friendly models and enemy models. When a friendly BCI speller and an enemy BCI speller coexist, an adversarial example may be mistakenly taken by the enemy BCI speller to signal an incorrect intention on the part of the person with a disability; in another scenario, an enemy BCI speller may leak personal information of the individual with a disability or initiate unwanted financial transactions. In this paper, we propose a method to create such a sample, called a "friend-guard" EEG adversarial example. In the proposed method, a very small EEG signal is added to an original sample, creating an adversarial example that the friendly model will classify correctly but the enemy model will classify incorrectly. A P300 dataset was used in the experimental evaluation of the method, and linear regression models were used as target models. In the experiment, the proposed method was able to generate friend-guard EEG adversarial examples that were incorrectly classified with success rates of 88.4% and 69.7% by the enemy model for subject A and subject B, respectively, while maintaining the accuracy of the friendly model at 85.9% and 74.4% for subjects A and B, respectively. [ABSTRACT FROM AUTHOR]

Recent research has shown that models induced by machine learning, and in particular by deep learning, can be easily fooled by an adversary who carefully crafts imperceptible, at least from the human perspective, or physically plausible modifications of the input data. This discovery gave birth to a new field of research, the adversarial machine learning, where new methods of attacks and defense are developed continuously, mimicking what is happening from a long time in cybersecurity. In this paper we will show that the drawbacks of inducing models from data less prone to be misled can actually provide some benefits when it comes to assessing their generalization abilities. We will show these benefits both from a theoretical perspective, using state-of-the-art statistical learning theory, and both with practical examples. [ABSTRACT FROM AUTHOR]

• The recent edge detection methods are classified and introduced in detail. • The existing widely used datasets and evaluation criteria for edge detection methods are reviewed. • Future research directions for edge detection are elaborated. Edge detection is one of the most important and fundamental problems in the field of computer vision and image processing. Edge contours extracted from images are widely used as critical cues for various image understanding tasks such as image segmentation, object detection, image retrieval, and corner detection. The purpose of this paper is to review the latest developments on image edge detection. Firstly, the definition and properties of edges are introduced. Secondly, the existing edge detection methods are classified and introduced in detail. Thirdly, the existing widely used datasets and evaluation criteria for edge detection methods are summarized. Finally, future research directions for edge detection are elaborated. [ABSTRACT FROM AUTHOR]

• We propose GATE-CNN architecture for binary classification of brain tumor using MR (Magnetic Resonance) images. • We have implemented our model on three different datasets; all consists of MRI with a significant difference in number of images. • The first dataset consists of tumorous and non-tumorous MRIs, second consists of MRIs with glioma and pitutiary (types of brain tumor), and the third consists of cancerous and non-cancerous (severity of brain tumor) images. • For all three datasets we found better classification results than any of the existing state-of-the-art models. Early and accurate detection is a solitary precaution to overcome the brain tumor. Otherwise, it will result in a deadly disease. Brain tumor (BT) detection using magnetic resonance is an essential and challenging job in the medical domain. To combat the aggressive spreading of BT, clinical imaging (MRI and X-Ray) can be an appropriate method for diagnosis. In this paper, we applied attention based image classification, which is a new paradigm for obtaining improved accuracy than the state-of-the-art models. We propose a GATE-CNN (Graph Attention AutoEncoder-Convolution Neural Network) model in this work for the classification of BT. We calculate the attention values of the neighboring pixels on each and every pixel present in the graph then process the graph using GATE framework and the processed graph with attention values is then passed to CNN framework for generation of final output. Further, we apply Adamax optimizer to optimize the training hyperparameters of CNN. We perform the BT classification using the proposed method on three different datasets among benign and malignant BT for first dataset, glioma and pituitary for second dataset, and normal and abnormal BT images for third dataset; all the three datasets consists of MRI images. We then compare the proposed model with a variety of CNN (Deep-CNN, Multi-Input CNN, and basic CNN) models dealing with several types of medical imaging (thermal image, MRI, X-Ray, etc.). Our model results in the accuracy of 98.27% for first dataset, 99.83% for second dataset, and 98.78% for third dataset demonstrating preferable network performance than the already established state-of-the-art CNN models. [ABSTRACT FROM AUTHOR]

• The work proposes an empirical analysis of memory degradation in RNN networks. • Attention based architectures tend to not use the RNN memory. • Direct input attention allows the RNN to work without attention interference. • The conjectures were tested on eight different problems. This paper studies the memory mechanisms in recurrent neural architectures when attention models are included. Pure-attention models like Transformers are more and more popular as they tend to outperform models with recurrent connections in many different tasks. Our conjecture is that attention prevents the recurrent connections from transferring information properly between consecutive next steps. This conjecture is empirically tested using five different models, namely, a model without attention, a standard Loung attention model, a standard Bahdanau attention model, and our proposal to add attention to the inputs in order to fill the gap between recurrent and parallel architectures (for both Luong and Bahdanau attention models). Eight different problems are considered to assess the five models: a sequence-reverse copy problem, a sequence-reverse copy problem with repetitions, a filter sequence problem, a sequence-reverse copy problem with bigrams and four translation problems (English to Spanish, English to French, English to German and English to Italian). The achieved results reinforce our conjecture on the interaction between attention and recurrence. [ABSTRACT FROM AUTHOR]

Recently, deep learning technology has achieved remarkable progress in compressed video quality enhancement. However, the existing methods fail to consider the fact that the regions with different compression losses contain varied effective information. To address this issue, this paper proposes a Compression Loss-based Spatial-Temporal Attention (CLSTA) module that predicts the compression loss-based attention of each pixel when restoring target frames. Based on this, the pixels with lower compression losses contribute more to restoring target frames. Meanwhile, the compression-domain information in the compressed video stream is exploited to overcome the difficulty in inferring compression loss-based attention directly from the pixel-domain. With a slight modification, the CLSTA module can be easily integrated into the existing methods for end-to-end training. A series of experiments have been conducted in this study to validate the effectiveness and generality of the CLSTA module for compressed video quality enhancement. By adding the CLSTA module to the state-of-the-art method STDF, the proposed method achieves an average PSNR improvement of 0.96 dB and a BD-rate reduction of 26.85% under the Low-Delay P configuration, which is respectively 0.06 dB and 1.83% better than the baseline STDF. In the best case, the proposed method obtains 0.19 dB higher PSNR than STDF for a single frame. [ABSTRACT FROM AUTHOR]

In continual learning over deep neural networks (DNNs), the rehearsal strategy , in which the previous exemplars are jointly trained with new samples, is commonly employed for the purpose of addressing catastrophic forgetting. Unfortunately, due to the memory limit, rehearsal-based techniques inevitably cause the class imbalance issue leading to a DNN biased toward new tasks having more samples. Existing works mostly focus on correcting such a bias in the fully connected layer or classifier. In this paper, we newly discover that class imbalance tends to make old classes even more highly correlated with their similar new classes in the feature space, which turns out to be the major reason behind catastrophic forgetting, called inter-task forgetting. To alleviate inter-task forgetting, we propose a novel class incremental learning method, called attractive & repulsive training (ART) , which effectively captures the previous feature space into a set of class-wise flags , and thereby makes old and new similar classes less correlated in the new feature space. In our empirical study, our ART method is observed to be quite effective to improve the performance of the state-of-the-art methods by substantially mitigating inter-task forgetting. Our implementation is available at: https://github.com/bigdata-inha/ART/. [ABSTRACT FROM AUTHOR]

Data cleaning is one of the most important tasks in data analysis processes. One of the perennial challenges in data analytics is the detection and handling of non-valid data. Failing to do so can result in creating imbalanced observations that can cause bias and influence estimates, and in extreme cases, can even lead to inaccurate analytics and unreliable decisions. Usually, the process of data cleaning is time-consuming due to its growing volume, velocity, and variety. Further, the complexity and difficulty of the cleaning process increase with the amount of data to be analyzed. It is rarely the case that any real-world data is clean and error-free. Thus, pre-processing the data before using it for analysis has become standard practice. This paper is intended to provide an easy-to-use and reliable system which automates the cleaning process for univariate time series data. Also, automating the process reduces the time required for cleaning it. Another issue that the proposed system aims to solve is making the visualization of a large amount of data more effective. To tackle these issues, an R package, cleanTS is proposed. The proposed system provides a way to analyze data on different scales and resolutions. Also, it provides users with tools and a benchmark system for comparing various techniques used in data cleaning. [ABSTRACT FROM AUTHOR]

[Display omitted] • This study is one of the pioneer studies combining soft sets and machine learning. • A new k NN-based FPFS- k NN has been proposed via distance measures of fpfs -matrices. • FPFS- k NN employs multi k -nearest neighbors based on pseudo-metrics of fpfs -matrices. • FPFS- k NN considers parameters' impact on classification. In this paper, we propose a new k NN algorithm, i.e., Fuzzy Parameterized Fuzzy Soft k NN (FPFS- k NN), based on multiple pseudo-metrics of fuzzy parameterized fuzzy soft matrices (fpfs -matrices). FPFS- k NN can consider the impacts of parameters on classification using pseudo-metrics of fpfs -matrices – a new concept. Furthermore, FPFS- k NN detects the nearest neighbors for each pseudo-metric and classifies data applying the aforementioned multiple distance functions. To demonstrate the classification success of the proposed method, we carry out an experimental study using 35 UCI datasets and comparing it with the state-of-the-art k NN-based and non- k NN-based algorithms. All the methods are trained and tested for ten runs through five-fold cross-validation. We then compare the results of FPFS- k NN with those of the others in terms of the most frequently used measures, such as accuracy (ACC), precision (PRE), recall (REC), micro F-score (MICF), and macro F-score (MACF). Afterward, we pro-vide a statistical evaluation of the results. Experimental and statistical results manifest that the proposed FPFS- k NN, utilized Pearson's correlation coefficient and denoted by FPFS- k NN (P) , outperforms the state-of-the-art k NN-based algorithms in 24 of 35 datasets in terms of each considered measure and 31 of 35 datasets in terms of accuracy measure. Besides, the results showed that FPFS- k NN (P) performs better than the others for 29 datasets in terms of ACC and MICF rates, and 24 datasets in terms of PRE, REC and MACF rates. Finally, we discuss FPFS- k NN for further research. [ABSTRACT FROM AUTHOR]

Algorithmic bias indicates the discrimination caused by algorithms, which occurs with protected features such as gender and race. Many researchers have tried to define the fairness and devise methods to mitigate bias, but it is still premature to reach the unanimous definition and evaluation metrics of fairness according to society, times and cultures. In this paper, we introduce three evaluation metrics such as parity difference, equalized opportunity and equalized odds that can deal with various definitions of algorithmic bias, and concretely divide the three general approaches further into seven methods with some challenges, resulting in relabeling, generation, fair representation (for pre-processing), constraint optimization, regularization (for in-processing), calibration and thresholding (for post-processing). Among them, the pre-processing method is widely used due to its versatility, but it has limitation to deal with the information on data and features related with bias appropriately. In order to preserve the characteristics of the original data while excluding the information about the features causing bias, we propose a preprocessing approach based on information theory that avoids collision in the dual optimization, where the latent space is divided into two subspaces. Experiments are conducted with the well-known benchmark datasets of Census and COMPAS, and two real-world tasks: facial emotion recognition and text sentiment analysis. The information theoretic approach is promising to achieve fair machine learning by reducing the bias caused by several features such as age, race and gender. [ABSTRACT FROM AUTHOR]

Class imbalance learning is one of the most important topics in the field of machine learning and data mining, and the Synthetic Minority Oversampling Techniques (SMOTE) is the common method to handle this issue. The main shortcomings of the classic SMOTE and its variants is the interpolation of potential noise and unrepresentative examples. This paper is devoted to proposing a novel parameter-free SMOTE mechanism to produce sufficient representative synthetic examples while avoiding interpolating noisy examples. Specifically, two types of minority class examples are defined, namely boundary and safe minority examples. The synthetic examples generation procedure fully reflects the characteristics of the minority class examples with filling the region dominated by the minority class and expanding the margin of the minority class. To verify the effectiveness and robustness of the proposed method, a thorough experimental study on forty datasets selected from real-world applications is carried out. The experimental results indicate that our proposed method is competitive to the classic SMOTE and its state-of-the-art variants. [ABSTRACT FROM AUTHOR]

The complexity of data has been increasing not only in the dimension of the data, but also in the interrelated class relationships. Pre-defined hierarchical structure of classes helps to model such relationships in a way, which divides the whole classification problem into a set of subtasks. Feature selection, known as a data preprocessing technique, aims to selecting useful feature to improve the performance of learning algorithms. However, the issue of close class relationships, especially under the same parent node in the hierarchical structure, is adverse to the feature selection. In this paper, we introduce a novel distance metric based feature selection method with large margin nearest neighbor strategy for the hierarchical classification problems. Different from other algorithms, our proposed method is to project the data into a low-dimensional feature space for learning correct metrics, and the weights of features will be indicated in the transformation matrix. A sparsity regularization and iterative optimization ensure that superior feature subsets can be selected. Experimental evaluation on several datasets with hierarchical class structure demonstrates that the proposed approach is comparable or better than some other state-of-the-art hierarchical feature selection methods. [ABSTRACT FROM AUTHOR]

[Display omitted] • Tackle the under-studied semi-supervised image regression problem to relief the heavy workload of manually annotation for regression tasks. • Applying manifold smoothness to image regression tasks that transfers the SSL problem to kernel learning tasks. • A novel and efficient algorithm, ManiDKL, learns the regression parameters in a Bayesian manner. • Outperforms all existing semi-supervised methods for image regression tasks. Semi-supervised learning (SSL) aims at utilizing the vast unlabeled data to help the supervised training. While existing SSL methods have shown promising results on image classification tasks, most of them rely on the cluster assumption that does not apply to image regression tasks. In this paper, we address the under-studied semi-supervised image regression problem, of which the outputs are continuous values instead of categorical distributions. To tackle this challenging task, we propose an algorithm, called ManiDKL, with the idea that the prediction function should be smooth with respect to the intrinsic manifold of data distribution and behave similarly on both labeled and unlabeled data. In particular, we propose a framework that implements the Tikhonov regularization with generative manifold learning to ensure manifold smoothness of regression function and also reduces the problem to kernel learning. Then a semi-supervised non-parametric Bayesian based deep kernel learning algorithm is proposed, in which unlabeled data are incorporated through posterior regularization. We show the effectiveness of ManiDKL with extensive experiments. It shows that ManiDKL performs comparatively with state-of-the-art SSL image classification methods. Most importantly, we show the superiority of ManiDKL over all existing SSL regression methods on public image datasets. [ABSTRACT FROM AUTHOR]

The target net method has been the foundation of deep reinforcement learning since Deepmind first proposed it in 2015. Almost all the current popular reinforcement learning algorithms include target net. However, while the slowly updated target network improves the stability of the algorithm, it also reduces the performance of the algorithm. In this paper, the authors design a novel triple-network algorithm(TPN). TPN combines the temporal-difference(TD) algorithm and policy gradient(PG) theorem. Using three networks to estimate the state value(v), action value (q) , and policy(π). These networks have no primary or secondary distinction but are trained synchronously and influence each other. The author found that through this TPN architecture, the convergence and stability of the algorithm can be greatly improved without increasing the amount of calculation. Although it is only a basic framework at present. The calculation process of TPN is simple and easy to implement. Experiments prove that the convergence speed and stability of TPN in discrete cases are better than PPO. • Proposed a new deep reinforcement learning algorithm that does not rely on target networks. • Presented the complete derivation process of the TPN algorithm. • Verified the effectiveness of proposed algorithm through experiments. [ABSTRACT FROM AUTHOR]

In this paper, an enhanced structural developmental neural network with information saturation (ESDNNIS) is proposed for continual unsupervised learning. It improves SDNNIS in the following respects: (1) it makes a distinction between the developmental stages of neurons to better regulate the updating of neurons; (2) it adopts the forgetting mechanism to control the network size; (3) it changes the way coverage domain parameter is initialized and updated to divide clusters more accurately; and (4) it adds a parent–child relationship update mechanism to provide better robustness to sample input order. ESDNNIS inherits the unsupervised incremental learning and class segmentation capabilities of SDNNIS, and achieves better class incremental clustering for more complex datasets while using fewer neurons than SDNNIS. The experimental results show that the accuracy of clustering results of ESDNNIS is improved by 10.94% on average compared with SDNNIS, while the number of generated neurons is reduced by 71.93% on average. [ABSTRACT FROM AUTHOR]

The Wasserstein distance in the optimal transport (OT) model is being widely used in machine learning applications due to its mathematical and geometric properties. Since the entropy regularized OT is a convex problem, and for its large-scale problems, many algorithms have been proposed. Existing methods mainly adopt iterative random projection techniques to approximate the Wasserstein distance in the OT model. In this paper, we employ the entropic domain decomposition algorithm with mini-batch learning to calculate the entropy regularized OT problem and investigate the stability of this algorithm. The algorithm performs on different cells of the domain instead of using the projection technique and it is based on a mini-batch loss function. The corresponding solution in each iteration is derived from the dual problem of entropy regularized OT on each decomposed cell. Once the solutions to the dual problem on all decomposed cells are obtained, the corresponding transport plan in each iteration can be obtained. We show that the transport plan computed by its dual variables is admissible, and we also give the sub-gradient approximation result. In addition, we exploit the statistical concentration result of the subsampled mini-batch Wasserstein distance on decomposed cells. Finally, the numerical experiments validate that the proposed algorithm can save computational time further. [ABSTRACT FROM AUTHOR]

Gradient-based algorithms are still popularly used for training radial basis function neural network (RBFNN). However, these algorithms may lead to the vanishing gradient (saddle point or local minimum) problem, which is one of the common problems that limit learning performance. To cope with this, a hybrid hyperplane gradient learning algorithm (HHGLA) is proposed to improve the learning performance of the RBFNN in this paper. First, a hyperplane gradient (HPG), based on a hyperspace constructed by a solution population (SP), is introduced. Thus, the search process can cross the surface of the cost function to avoid the appearance of vanishing gradient. Second, an adaptive learning rate is designed to restrict the search process into this hyperspace. Then, during the learning process, the hyperspace is constantly contracted to approximate the global optimal solution. Third, the convergence property of HHGLA-based RBFNN is thoroughly analyzed to ensure its application success. Finally, empirical comparisons of the proposed HHGLA-RBFNN are given to illustrate its superiority. The simulation results show the feasibility of HHGLA-RBFNN of accuracy. [ABSTRACT FROM AUTHOR]

Federated learning, a privacy-preserving collaborative machine learning paradigm, has led to the proposal of various incentive mechanisms to encourage active participation of data owners. However, most of the existing mechanisms focused on the monopsony market scenario, where only one server-side entity (buyer) is involved. In real-world scenarios, multiple server parties may express simultaneous interest in the data of a client (seller), leading to a non-monopoly market. This paper aims to bridge this gap by introducing the concept of incentivizing federated learning in a non-monopoly market and presents a non-monopoly federated learning incentive mechanism, coined as NmFLI. NmFLI employs a double-auction mechanism to implement federated learning incentives and utilizes the Vickery–Clarke–Groves (VCG) mechanism to ensure client trustworthiness. Additionally, NmFLI devises a method for measuring data quality by calculating the value of clients based on their historical performance, which effectively balances accuracy and computational complexity. We demonstrate that NmFLI possesses properties such as individual rationality and strategy-proofness. Experimental results indicate that NmFLI can effectively incentivize federated learning and achieve higher accuracy than baseline models across various scenarios. For example, when the objectives of various tasks overlap, NmFLI outperforms the best baseline by 3.09% with imbalanced client data while maintaining the same data size. Moreover, NmFLI surpasses the best baseline by 6.12% with different amounts of client data. [ABSTRACT FROM AUTHOR]

The expensive optimization problem, widespread in real-world applications, poses great challenges to traditional evolutionary algorithms (EAs) due to its costly evaluations. A lot of investigations have proved that it is a practical way to incorporate surrogate models into EAs to reduce the number of time-consuming evaluations. Although various types of surrogate models from the community of machine learning, such as regression models that approximate the fitness values of candidate solutions or classification models that predict candidate solutions' category, have been widely applied separately, none of them consistently outperforms the other. Considering the intrinsic consistency of alleviating computational burden to improve the optimization efficiency, this paper introduces a two-layer architecture to integrate regression and classification models and synergize global and local evolution for high-dimensional expensive many-objective optimization problem (EMaOP). To implement this idea, the Kriging model based on incremental learning is utilized to explore the potential promising space. Further, multi local support vector machine (SVM) classifiers assisted improved differential evolution is run by turn with the global search to accelerate convergence. Accordingly, a novel surrogate-assisted EA, named Classification- and Regression-assisted evolutionary algorithm (CR-SAEA), is proposed on this basis. Experimental studies on several benchmark functions with up to 11 objectives and 200 dimensions confirm the effectiveness of the proposed algorithm and its competitiveness against state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

Uncertainty estimation is a critical component of building safe and reliable machine learning models. Accurate estimation of uncertainties is essential for identifying and mitigating potential risks and ensuring that machine learning systems operate reliably in real-world scenarios. Various approaches, such as ensemble and Bayesian neural networks have been developed by sampling probability predictions from submodels, which is computationally expensive. At present, these techniques are incapable of precisely delineating the boundary separating in-distribution (ID) and out-of-distribution (OOD) data. To fill up this research gap, this paper presents a normalizing flow based framework to directly predict parameters of prior distributions over the probability with a neural network, the proposed model is able to effectively differentiate between ID and OOD data in regression problems. The posterior distributions learned by the model precisely represent uncertainties for OOD data based solely on ID data, without the need for OOD data during training. This approach has shown promising results in a number of applications, including image depth estimation and image adversarial attacks. [ABSTRACT FROM AUTHOR]

Effective animal intelligent monitoring is of great value in terms of ecological protection and endangered specie conservation. At present, computer vision technologies have shed light on animal intelligent monitoring. Especially, numerous deep learning-based models and methods have been developed to address various challenges, and have made substantial strides in this field. However, there are still several problems to be solved and related areas to be mined, such as exploring new strategies to enhance the robustness and generalization ability of models, designing novel models for complex environments, and establishing large-scale publicly available animal datasets for performance verification of models. Therefore, we comprehensively elaborated and analyzed existing works on animal intelligent monitoring based on advanced information science technologies, so as to provide useful information assistance for relevant researchers. In this paper, we focus on three primary task fields: precise animal localization, tracking and individual identification. Specifically, we elucidate the definition and significance of each monitoring task, and summarize the baseline models for addressing different problems. We provide a specific analysis of strategies and prototypes of the models and methods employed in each tasks following by the technical progression from traditional machine learning to deep learning. In addition, we make a comparison and analysis of the relevant methods, summarize their similarities and differences between them, and point out the advantages and disadvantages of these methods. Finally, we present several unresolved challenges and problems in animal intelligent monitoring and provide potential research directions in the future. We expect that our review can serve as reference and guidance for related research fields. [ABSTRACT FROM AUTHOR]

The eight papers included in this special issue represent a selection of extended contributions presented at the 17th International Conference on Soft Computing Models in Industrial and Environmental Applications, SOCO 2022 held in Salamanca, Spain, September 6th-8th, 2022, and organized by the BISITE group at University of Salamanca. SOCO 2022 international conference represents a collection or set of computational techniques in machine learning, computer science and some engineering disciplines which investigate, simulate, and analyse very complex issues and phenomena. This special issue is aimed at practitioners, researchers, and postgraduate students who are engaged in developing and applying advanced intelligent systems principles to solve real-world problems in the mentioned fields. [ABSTRACT FROM AUTHOR]

Addressing the reproducibility crisis in artificial intelligence through the validation of reported experimental results is a challenging task. It necessitates either the reimplementation of techniques or a meticulous assessment of papers for deviations from the scientific method and best statistical practices. To facilitate the validation of reported results, we have developed numerical techniques capable of identifying inconsistencies between reported performance scores and various experimental setups in machine learning problems, including binary/multiclass classification and regression. These consistency tests are integrated into the open-source package mlscorecheck , which also provides specific test bundles designed to detect systematically recurring flaws in various fields, such as retina image processing and synthetic minority oversampling. [ABSTRACT FROM AUTHOR]

Neural networks and deep learning algorithms are designed to function similarly to biological synaptic structures. However, classical deep learning algorithms fail to fully capture the need for continuous learning; this has led to the advent of incremental learning. Incremental learning adds new challenges that are handled differently by modern state-of-the-art approaches. Some of these include: utilization of network memory as additional knowledge increases the size of the network, open-set recognition to be able to identify unrecognized information, and efficient knowledge distillation as most incremental learning algorithms are prone to catastrophic forgetting of previously learned knowledge. Recent advancements achieve incremental learning through a multitude of methods. Most methods are characterized by augmenting the normal algorithm of neural network training by both directly modifying the neural network structure and by adding additional learning steps. This paper analyzes and provides a comprehensive survey of existing methods and various techniques used for incremental learning. A novel categorization of the methods is also introduced based on recent trends of the state-of-the-art solutions. The study focuses on methods that provide incremental learning success as well as discusses emerging patterns in new research. [ABSTRACT FROM AUTHOR]

A-to-I RNA editing is a long-known driving factor to the progression of hepatocellular carcinoma (HCC); however, its importance is often neglected in HCC subtyping and the modeling of prognosis, partially due to the high dimensional nature of the editing data. In this paper, we first obtain a comprehensive A-to-I RNA editing profile for HCC, among which two editing sites, IGFBP7 I387M (chr4:57110120) and SUMF2 A98G (chr7:56078974), are identified as having the driving function potential. Then, we exploit the clinical-related RNA editing profile in combination with the gene expressions to develop a new prognostic model for HCC. To preserve the inner structure of this editing profile in a latent space that has a much lower dimension, we employ variational autoencoder (VAE), an unsupervised learning neural network applied for the first time to compress the RNA editing profile. Empowered by VAE, we are able to cluster the HCC patients efficiently according to their editing profile, among which a group of patients with immune dysregulation and enrichment in cell proliferation and cancer-related pathways are identified. Based on the editing-related subtyping, we develop a prognostic model that consists of an easy-computing risk score and an adaptively trained threshold. Validations using three independent cohorts (LIHC, LIRI, and GSE14520) confirm that the new model is robust, sensitive, and specific. Furthermore, it is more accurate in prediction than six state-of-the-art competitors. The model is also found applicable to 18 more cancers besides HCC, which suggests a strong ability in generalization. To facilitate clinical usage, we develop a nomogram, drug targets, prognosis-associated genes, and relevant pathways are also identified, which pave the way for upcoming studies on HCC's molecular mechanism, therapeutic targets, and many others. • A New prognostic modeling approach incorporating the A-to-I RNA editing information into classical framework featured by genes. • A prognostic model for HCC using the approach validated in 3 cohorts and outperforming 6 state-of-the-art competitors. • Two editing sites, IGFBP7 I387M (chr4:57110120) and SUMF2 A98G (chr7:56078974), located and found associated with HCC. [ABSTRACT FROM AUTHOR]

This paper proposes a novel recurrent context layered radial basis function neural network (RCLRBFNN) for the identification of nonlinear dynamical systems. The proposed model consists of an additional context layer in which the nodes represent the unit-delayed outputs of the hidden layer radial centers. These delayed outputs undergo a nonlinear transformation by applying a tangent hyperbolic function. These transformed signals connect to the output layer neuron through the adjustable context layered weights. To tune the parameters of the proposed model the update equations are derived using the dynamic back-propagation algorithm. Further, an adaptive learning rate scheme is proposed to improve the performance of the learning algorithm. In the simulation experiment, a total of two examples are considered to test the efficacy and performance of the proposed model. The performance comparison is made with the conventional structure of the radial basis function neural network (RBFNN), Jordan Recurrent neural network (JRNN), and the feed-forward neural network (FFNN) (which is nothing but a single-layered multi-layered perceptron). Both the disturbance signal as well as system's uncertainty scenarios are considered to test the robustness shown by the proposed model. The results showed that the proposed model has delivered a better identification accuracy as compared to the other neural models. [ABSTRACT FROM AUTHOR]

Human intention prediction is of great significance in many applications, such as human-robot interaction, intelligent rehabilitation robots. This paper studies the problem of short-term next-active-object prediction in egocentric images. The short-term next-active-object refers to the object that a human is going to interact with in the short-term future, which is an embodiment of human intention. Most current methods usually use object-centered cues, such as the deviation of object appearance change and the unique shape of the egocentric object trajectory, to predict the next-active-object. In this paper, inspired by the fact that human intention is also revealed by human-centered cues, we propose a deep neural network model that integrates the cues from visual attention and hand positions to predict the next-active-object. Firstly, the probability maps of visual attention and hand positions are constructed, and then the probability distribution of next-active-object is generated. We experimentally compare our method with several baseline methods using two datasets and confirm its effectiveness. In addition, ablation experiments are conducted, and crucial points concerning the next-active-object are discussed. [ABSTRACT FROM AUTHOR]