1,649 results
Search Results
2. Joint coupled representation and homogeneous reconstruction for multi-resolution small sample face recognition.
- Author
-
Fan, Xiaojin, Liao, Mengmeng, Xue, Jingfeng, Wu, Hao, Jin, Lei, Zhao, Jian, and Zhu, Liehuang
- Subjects
- *
MACHINE learning , *FRACTIONAL programming , *FACE perception , *SHOOTING equipment , *LEARNING - Abstract
• This paper proposes a novel multivariate dictionary learning framework. • A coherence enhancement term to improve the coherent representing of the coding coefficients under different resolutions. • A multivariate dictionary optimization method to solve dictionaries involving the calculation of fractional norm. • The proposed method achieves the state-of-the-art performance on several benchmark datasets. Off-the-shelf dictionary learning algorithms have achieved satisfactory results in small sample face recognition applications. However, the achieved results depend on the facial images obtained at a single resolution. In practice, the resolution of the images captured on the same target is different because of the different shooting equipment and different shooting distances. These images of the same category at different resolutions will pose a great challenge to these algorithms. In this paper, we propose a Joint Coupled Representation and Homogeneous Reconstruction (JCRHR) for multi-resolution small sample face recognition. In JCRHR, an analysis dictionary is introduced and combined with the synthetic dictionary for coupled representation learning, which better reveals the relationship between coding coefficients and samples. In addition, a coherence enhancement term is proposed to improve the coherent representation of the coding coefficients at different resolutions, which facilitates the reconstruction of the sample by its homogeneous atoms. Moreover, each sample at different resolutions is assigned a different coding coefficient in the multi-dictionary learning process, so that the learned dictionary is more in line with the actual situation. Furthermore, a regularization term based on the fractional norm is drawn into the dictionary coupled learning to remove the redundant information in the dictionary, which can reduce the negative impacts of the redundant information. Comprehensive results demonstrate that the proposed JCRHR method achieves better results than the state-of-the-art methods, on several small sample face databases. [ABSTRACT FROM AUTHOR]
- Published
- 2023
- Full Text
- View/download PDF
3. Special issue on selected and extended papers from the 2015 International Conference on Intelligence Science and Big Data Engineering (IScIDE 2015).
- Author
-
Shan, Shiguang, Chang, Hong, Cai, Deng, and Deng, Cheng
- Subjects
- *
IMAGE processing , *COMPUTER vision , *MACHINE learning - Published
- 2017
- Full Text
- View/download PDF
4. A survey for solving mixed integer programming via machine learning.
- Author
-
Zhang, Jiayi, Liu, Chang, Li, Xijun, Zhen, Hui-Ling, Yuan, Mingxuan, Li, Yawen, and Yan, Junchi
- Subjects
- *
MACHINE learning , *INTEGER programming , *COMBINATORIAL optimization , *HEURISTIC algorithms , *NP-hard problems , *MACHINE theory , *PROBLEM solving - Abstract
Machine learning (ML) has been recently introduced to solving optimization problems, especially for combinatorial optimization (CO) tasks. In this paper, we survey the trend of leveraging ML to solve the mixed-integer programming problem (MIP). Theoretically, MIP is an NP-hard problem, and most CO problems can be formulated as MIP. Like other CO problems, the human-designed heuristic algorithms for MIP rely on good initial solutions and cost a lot of computational resources. Therefore, researchers consider applying machine learning methods to solve MIP since ML-enhanced approaches can provide the solution based on the typical patterns from the training data. Specifically, we first introduce the formulation and preliminaries of MIP and representative traditional solvers. Then, we show the integration of machine learning and MIP with detailed discussions on related learning-based methods, which can be further classified into exact and heuristic algorithms. Finally, we propose the outlook for learning-based MIP solvers, the direction toward more combinatorial optimization problems beyond MIP, and the mutual embrace of traditional solvers and ML components. We maintain a list of papers that utilize machine learning technologies to solve combinatorial optimization problems, which is available at https://github.com/Thinklab-SJTU/awesome-ml4co. [ABSTRACT FROM AUTHOR]
- Published
- 2023
- Full Text
- View/download PDF
5. Neural-based fixed-time composite learning control for multiagent systems with intermittent faults.
- Author
-
Zheng, Xiaohong, Ren, Hongru, Zhou, Qi, and Wang, Xinzhong
- Subjects
- *
MACHINE learning , *FAULT-tolerant control systems , *CLOSED loop systems , *NONLINEAR functions , *NONLINEAR systems - Abstract
In this paper, a distributed fixed-time composite learning control problem is addressed for nonlinear multiagent systems (MASs) subject to intermittent actuator faults. First, a distributed estimator is constructed for followers that are unable to communicate directly with the leader. Then, instead of using the traditional adaptive neural network (NN) algorithm, a predictor-based composite learning technique is proposed, which incorporates the prediction error into the NN update law to enhance the estimation accuracy of the unknown nonlinearity. Furthermore, an adaptive fault-tolerant control compensation mechanism is developed for intermittent faults that may occur indefinitely and frequently. To guarantee that all signals of the closed-loop system are bounded in fixed time, a nonsingular fixed-time fault-tolerant controller in the form of quadratic function is established. Finally, simulation results confirm the effectiveness of the presented algorithm. • This paper presents a singularity-free fixed-time NN algorithm for nonlinear MASs, and a composite learning algorithm is established to improve the approximation accuracy of nonlinear functions by introducing a prediction error into NN update law. • For followers without access to the leader, a local estimator is utilized to estimate the leader information. Therefore, the present control method avoids the emergence of coupling terms between agents during the controller design. • This paper considers intermittent actuator faults that may occur indefinitely and frequently, posing significant challenges. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
6. A review of research on reinforcement learning algorithms for multi-agents.
- Author
-
Hu, Kai, Li, Mingyang, Song, Zhiqiang, Xu, Keer, Xia, Qingfeng, Sun, Ning, Zhou, Peng, and Xia, Min
- Subjects
- *
MACHINE learning , *REWARD (Psychology) , *ARTIFICIAL intelligence , *LITERATURE reviews , *MULTIAGENT systems , *REINFORCEMENT learning - Abstract
In recent years, multi-agent reinforcement learning techniques have been widely used and evolved in the field of artificial intelligence. However, traditional reinforcement learning methods have limitations such as long training time, large sample data requirements, and highly delayed rewards. Therefore, this paper systematically and specifically studies the MARL algorithm. Firstly, this paper uses Citespace software to visually analyze the existing literature on multi-agent reinforcement learning and briefly indicates the research hotspots and key research directions in this field. Secondly, the applications of traditional reinforcement learning algorithms under two task objects, namely single-agent and multi-agent systems, are described in detail. Then, the paper highlights the diverse applications, challenges, and corresponding solutions of MARL algorithmic techniques in the field of MAS. Finally, the paper points out future research directions based on the existing limitations of the algorithm. Through this paper, readers will gain a systematic and in-depth understanding of MARL algorithms and how they can be utilized to better address the various challenges posed by MAS. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
7. Exploiting indirect linear correlation for label distribution learning.
- Author
-
Yu, Peiqiu and Jia, Xiuyi
- Subjects
- *
MACHINE learning , *MATRIX decomposition , *LABEL design , *ALGORITHMS , *HYPOTHESIS - Abstract
Label distribution learning represents the relevance of labels to samples using description degree, which can provide richer semantic information, thus finding wider applications. Exploiting label correlations is an effective approach to narrow down the hypothesis space of label distribution learning models. In existing works that utilize low-rank assumptions or label linear dependence to mine correlations, it is assumed that a label can be linearly expressed by other labels. However, this assumption can only be satisfied when there are linear dependency relationships between labels, thus the label correlation obtained by such methods is subject to certain distortion. To address this issue, this paper assumes that labels can be linearly represented by the same set of bases. The correlation between labels is represented by sharing common bases. Specifically, the paper employs matrix factorization to extract bases that can be used to represent all labels. And then designs a label distribution learning algorithm based on the property of sharing the same set of bases of the ground truth label distribution and predict label distribution. The effectiveness of the algorithm is verified through experimental validation. Generally speaking, the algorithm presented in this paper achieves optimal performance at 73.15% of the cases, with the best average ranking. In the two-tailed t -test, the algorithm in this paper exhibits statistical superiority compared to all comparison algorithms. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
8. Deep learning for Covid-19 forecasting: State-of-the-art review.
- Author
-
Kamalov, Firuz, Rajab, Khairan, Cherukuri, Aswani Kumar, Elnagar, Ashraf, and Safaraliev, Murodbek
- Subjects
- *
DEEP learning , *COVID-19 , *COVID-19 pandemic , *FORECASTING , *MACHINE learning , *QUALITY control - Abstract
• The paper fills the gap in the literature by reviewing and analyzing the current studies that apply deep learning for Covid-19 forecasting. • The initial search identified 152 studies of which 53 passed the quality control. • The existing literature is categorized using a model-based taxonomy. • The description of the models along with their performance evaluation is presented. • Recommendations for future improvements are provided. The Covid-19 pandemic has galvanized scientists to apply machine learning methods to help combat the crisis. Despite the significant amount of research there exists no comprehensive survey devoted specifically to examining deep learning methods for Covid-19 forecasting. In this paper, we fill the gap in the literature by reviewing and analyzing the current studies that use deep learning for Covid-19 forecasting. In our review, all published papers and preprints, discoverable through Google Scholar, for the period from Apr 1, 2020 to Feb 20, 2022 which describe deep learning approaches to forecasting Covid-19 were considered. Our search identified 152 studies, of which 53 passed the initial quality screening and were included in our survey. We propose a model-based taxonomy to categorize the literature. We describe each model and highlight its performance. Finally, the deficiencies of the existing approaches are identified and the necessary improvements for future research are elucidated. The study provides a gateway for researchers who are interested in forecasting Covid-19 using deep learning. [ABSTRACT FROM AUTHOR]
- Published
- 2022
- Full Text
- View/download PDF
9. Robust learning of Huber loss under weak conditional moment.
- Author
-
Huang, Shouyou
- Subjects
- *
STATISTICAL learning , *MACHINE learning - Abstract
In this paper, we study the performance of robust learning with Huber loss. As an alternative to traditional empirical risk minimization schemes, Huber regression has been extensively used in machine learning. A new comparison theorem is established in the paper, which characterizes the gap between the excess generalization error and the prediction error. In addition, we refine the error bounds from the perspective of statistical learning theory and improve the convergence rates in the presence of heavy-tailed noise. It is worth mentioning that a new moment condition E [ | Y | 1 + ∊ | X = x ] ∈ L ρ X 2 is employed in analysis of error bound and learning rates from a theoretical viewpoint. [ABSTRACT FROM AUTHOR]
- Published
- 2022
- Full Text
- View/download PDF
10. Efficient hyperspectral image segmentation for biosecurity scanning using knowledge distillation from multi-head teacher.
- Author
-
Phan, Minh Hieu, Phung, Son Lam, Luu, Khoa, and Bouzerdoum, Abdesselam
- Subjects
- *
MACHINE learning , *BIOSECURITY , *IMAGE segmentation , *TEACHERS' assistants , *TEACHERS , *PERFORMANCE standards - Abstract
Foreign species can deteriorate the environment and the economy of a country. To automatically monitor biosecurity threats at country borders, this paper investigates compact deep networks for accurate and real-time object segmentation for hyperspectral images. To this end, knowledge distillation (KD) approaches compress the model by distilling the knowledge of a large teacher network to a compact student network. However, when the student is over-compressed, the performance of standard KD methods degrades significantly due to the large capacity gap between the teacher and the student. This gap can be addressed by adding medium-sized teacher assistants, but training them incurs significant computation and hence is impractical. To address this problem, this paper proposes a new framework called Knowledge Distillation from Multi-head Teacher (KDM), which distills the knowledge of a multi-head teacher to the student. By encapsulating multiple teachers in a single network, our proposed KDM assists the learning of a very compact student and significantly reduces the training time. We also introduce Bio-HSI, a new large benchmark hyperspectral image dataset of 3,125 high-resolution images with dense segmentation ground truth. This new, large dataset can be expected to advance research on deep models for hyperspectral image segmentation. Evaluated on this dataset, the student trained via our KDM has 762 times fewer parameters than the state-of-the-art segmentation model (i.e., HRNet), while achieving competitive accuracy. [ABSTRACT FROM AUTHOR]
- Published
- 2022
- Full Text
- View/download PDF
11. A deep reinforcement learning based method for real-time path planning and dynamic obstacle avoidance.
- Author
-
Chen, Pengzhan, Pei, Jiean, Lu, Weiqing, and Li, Mingzhen
- Subjects
- *
REWARD (Psychology) , *MACHINE learning , *REINFORCEMENT learning , *STATISTICAL sampling - Abstract
In a dynamic environment, the moving obstacle makes the path planning of the manipulator very difficult. Therefore, this paper proposes a path planning with dynamic obstacle avoidance method of the manipulator based on a deep reinforcement learning algorithm soft actor-critic (SAC). To avoid the moving obstacle in the environment and make real-time planning, we design a comprehensive reward function of dynamic obstacle avoidance and target approach. Aiming at the problem of low sample utilization caused by random sampling, in this paper, prioritized experience replay (PER) is employed to change the weight of samples, and then improve the sampling efficiency. In addition, we carry out the simulation experiment and give the results. The result shows that this method can effectively avoid moving obstacles in the environment, and complete the planning task with a high success rate. [ABSTRACT FROM AUTHOR]
- Published
- 2022
- Full Text
- View/download PDF
12. Perspective view of autonomous control in unknown environment: Dual control for exploitation and exploration vs reinforcement learning.
- Author
-
Chen, Wen-Hua
- Subjects
- *
REINFORCEMENT learning , *MACHINE learning , *PROBLEM solving , *DECISION making , *ACTIVE learning - Abstract
This paper overviews and discusses the relationship between Reinforcement Learning (RL) and the recently developed Dual Control for Exploitation and Exploration (DCEE). It is argued that there are two related but quite distinctive approaches, namely, control and machine learning, in tackling intractability arising in optimal decision making/control problems. In the control approach, the original problems (of an infinite horizon) are approximated by finite horizon problems and solved online by taking advantage of the availability of computing power. In the machine learning approach, the optimal solutions are approximated through iterations, or (offline) training through trials when models are not available. When dealing with unknown environments, DCEE as a technique developed from the control approach could potentially solve similar problems as RL while offering a number of advantages, most notably, coping with uncertainty in environment/tasks, high efficiency in learning through balancing exploitation and exploration, and potential in establishing its formal properties like stability. The links between DCEE and other relevant methods like dual control, Model Predictive Control and particularly Active Inference in neuroscience are discussed. The latter provides a strong biological endorsement for DCEE. The methods and discussions are illustrated by autonomous source search using a robot. It is concluded that DCEE provides a promising, complementary approach to RL, and more research is required to develop it as a generic theory and fully realise its potential. The relationships revealed in this paper provide insights into these relevant methods and facilitate cross fertilisation between control, machine learning and neuroscience for developing autonomous control under uncertain environments. [ABSTRACT FROM AUTHOR]
- Published
- 2022
- Full Text
- View/download PDF
13. EvoSTGAT: Evolving spatiotemporal graph attention networks for pedestrian trajectory prediction.
- Author
-
Tang, Haowen, Wei, Ping, Li, Jiapeng, and Zheng, Nanning
- Subjects
- *
PEDESTRIANS , *VIDEO surveillance , *SOCIAL influence , *INFORMATION modeling , *MACHINE learning , *SOCIAL interaction - Abstract
[Display omitted] • This paper addresses the problem of pedestrian trajectory prediction, which plays significant roles in many applications such as human-robot cooperation and video surveillance. • It proposes an evolving spatiotemporal graph attention network model, which employs a time-varying graph convolution to extract time-varying features and an evolving attention mechanism to describe the recursive temporal interactions. • It improves the pedestrian trajectory predicting results and proves the strength of the model with ablation studies. Predicting pedestrian trajectory is an essential task in many applications. While previous studies based on graphs seek to model spatiotemporal information among pedestrian interactions, most of them neglect the recursive and continuous relations between neighboring time points. In this paper, we propose an evolving spatiotemporal graph attention network to predict future trajectories of pedestrians. This model considers the evolving relations of social interactions between contiguous time points and uses coordinates. The interaction is modeled by an evolving and dynamic attention mechanism. The social influence of each pedestrians of current frame is evolved from that of last frame and will be utilized to generate the social influence of next frame. The proposed model was tested on two challenging datasets and the experimental results prove the strength of the model. [ABSTRACT FROM AUTHOR]
- Published
- 2022
- Full Text
- View/download PDF
14. Analysis methods of coronary artery intravascular images: A review.
- Author
-
Huang, Chenxi, Wang, Jian, Xie, Qiang, and Zhang, Yu-Dong
- Subjects
- *
DEEP learning , *CORONARY arteries , *HEART disease diagnosis , *INTRAVASCULAR ultrasonography , *OPTICAL coherence tomography , *IMAGE analysis , *MACHINE learning - Abstract
Coronary artery disease is among one of the diseases human suffer most. Intravascular coronary arterial image analysis consists of denoising, segmentation, detection, and three-dimensional reconstruction, having a significant meaning for auxiliary diagnosis and treatment of coronary artery disease. Intravascular ultrasound (IVUS) and intravascular optical coherence tomography (IVOCT) are the two most commonly applied intravascular coronary arterial imaging techniques. Based on these fundamental imaging techniques, in recent years, many advanced technologies from traditional machine learning algorithms to deep learning methods were employed in the analysis of intravascular coronary arterial images and made huge progress in this field. In this survey, we reviewed more than one hundred papers published in top journals or conferences such as Neural Networks and MICCAI. These papers proposed approaches or schemes for the intravascular coronary arterial image analysis, including lumen border segmentation, atherosclerotic plaque characterization, media-adventitia segmentation, stent strut detection, and three-dimensional reconstruction. Our survey began with introducing coronary artery intravascular imaging techniques, essential neural networks, and deep learning and then presented an across-the-board review of methods, applications, and trends of intravascular image analysis. This survey is more comprehensive than other articles not only for its scope and reference number but also for discussing the future direction in this field. Compared to other review papers in this field, this article could assist beginners in constructing a basic knowledge frame of coronary artery intravascular image analysis methods and brought state-of-the-art progress in this field to fellow researchers. We hope this paper could benefit either the beginners for coronary arterial image analysis or experienced researchers. [ABSTRACT FROM AUTHOR]
- Published
- 2022
- Full Text
- View/download PDF
15. A review of regression and classification techniques for analysis of common and rare variants and gene-environmental factors.
- Author
-
Miller, Anthony, Panneerselvam, John, and Liu, Lu
- Subjects
- *
GENOME-wide association studies , *GENOTYPE-environment interaction , *MACHINE learning , *TYPE 1 diabetes , *GENOMICS - Abstract
Statistical techniques incorporated with machine-learning algorithms in unison with gene-environment interaction are giving unparalleled understanding of complex diseases. Accurate analysis and intricate capturing of common, rare, and low MAF (Minor Allele Frequency) variants alongside gene-environmental interaction is pivotal whilst concluding reliable and accurate classification of complex diseases. Various complex diseases including genres of diabetes Type 1 and Type 2 alongside the vastly under-researched Lada (Latent Autoimmune Diabetes in Adults) diabetes require further investigation alongside significant machine learning research to gain a deeper understanding of the disease complexities. Despite existing efforts, an ideal combination of statistical techniques with optimal machine-learning algorithms that can accurately capture and model the gene-environment interaction is lacking. Intentionally exploring future and simultaneously exploiting modern-day computational methods in genomic analysis, this paper profoundly investigates both the future and present interaction of statistical analysis techniques and machine-learning algorithms and Ensembles with gene-environmental factors. In this context, this paper firstly presents a conceptual understanding of genomic conventions; secondly, conducts potential future machine learning algorithms alongside an extensive analysis of a range of classification, regression and Ensemble techniques along with exhibiting their imperative relationship and roles in investigating and classifying common, rare variants and a wide array of gene-environmental factors; and thirdly, utilisation of statistical techniques in Genome Wide Association Studies is scrutinised whilst analysing common, rare and MAF variants. As an important contribution, this paper identifies efficient machine-learning algorithms alongside Ensemble models and future potential analysis techniques and exhibits their inherent characteristics that can enhance the reliability and accuracy of the gene-environment classification analysis. [ABSTRACT FROM AUTHOR]
- Published
- 2022
- Full Text
- View/download PDF
16. Noise/fault aware regularization for incremental learning in extreme learning machines.
- Author
-
Wong, Hiu-Tung, Leung, Ho-Chun, Leung, Chi-Sing, and Wong, Eric
- Subjects
- *
MACHINE learning , *NOISE , *NONLINEAR regression , *FAULT tolerance (Engineering) , *NONLINEAR equations , *FAULT-tolerant computing , *FAULT diagnosis - Abstract
• This paper develops a noise/fault aware training objective for incremental ELM. • This paper uses two representative algorithms to develop two noise aware ELM. • The two proposed algorithms are much better than existing ones. • The multiple set concept can further enhance the performance. • We can make other non-noise tolerant algorithms to be noise tolerant. This paper investigates noise/fault tolerant incremental algorithms for the extreme learning machine (ELM) concept. Existing incremental ELM algorithms can be classified into two approaches: non-recomputation and recomputation. This paper first formulates a noise/fault aware objective function for nonlinear regression problems. Instead of developing noise/fault aware algorithms for the two computational approaches in a one-by-one manner, this paper uses two representative incremental algorithms, namely incremental ELM (I-ELM) and error minimized ELM (EM-ELM), to develop two noise/fault aware incremental algorithms. The proposed algorithms are called generalized I-ELM (GI-ELM) and generalized EM-ELM (GEM-ELM). The GI-ELM adds k hidden nodes into the existing network at each incremental step without recomputing the existing weights. To have a fair comparison, we consider a modified version of I-ELM as a comparison algorithm. The simulation demonstrates that the noise/fault tolerance of the proposed GI-ELM is better than that of the modified I-ELM. In the GEM-ELM, k hidden nodes are added into the existing network at each incremental step. Meanwhile, all output weights are recomputed based on a recursive formula. We also consider a modified version of EM-ELM as a comparison algorithm. The simulation demonstrates that the noise/fault tolerance of the proposed GEM-ELM is better than that of the modified EM-ELM. Moreover, we demonstrate that the multiple set concept can further enhance the performance of the two proposed algorithms. Following our research results, one can make some non-noise/fault tolerant incremental algorithms to be noise/fault tolerant. [ABSTRACT FROM AUTHOR]
- Published
- 2022
- Full Text
- View/download PDF
17. Neurocomputing for internet of things: Object recognition and detection strategy.
- Author
-
Qureshi, Kashif Naseer, Kaiwartya, Omprakash, Jeon, Gwanggil, and Piccialli, Francesco
- Subjects
- *
OBJECT recognition (Computer vision) , *INTERNET of things , *ARTIFICIAL intelligence , *SMART devices , *MACHINE learning - Abstract
Modern and new integrated technologies have changed the traditional systems by using more advanced machine learning, artificial intelligence methods, new generation standards, and smart and intelligent devices. The new integrated networks like the Internet of Things (IoT) and 5G standards offer various benefits and services. However, these networks have suffered from multiple object detection, localization, and classification issues. Conventional Neural Networks (CNN) and their variants have been adopted for object detection, classification, and localization in IoT networks to create autonomous devices to make decisions and perform tasks without human intervention and helpful to learn in-depth features. Motivated by these facts, this paper investigates existing object detection and recognition techniques by using CNN models used in IoT networks. This paper presents a Conventional Neural Networks for 5G-Enabled Internet of Things Network (CNN-5GIoT) model for moving and static objects in IoT networks after a detailed comparison. The proposed model is evaluated with existing models to check the accuracy of real-time tracking. The proposed model is more efficient for real-time object detection and recognition than conventional methods. [ABSTRACT FROM AUTHOR]
- Published
- 2022
- Full Text
- View/download PDF
18. Adaptive Regularized Warped Gradient Descent Enhances Model Generalization and Meta-learning for Few-shot Learning.
- Author
-
Rao, Shuzhen, Huang, Jun, and Tang, Zengming
- Subjects
- *
MACHINE learning - Abstract
Warped Gradient descent (WarpGrad) is a remarkable meta-learning method for gradient transformation by inserting warp-layers. However, the task-shared initialization provided by WarpGrad is difficult to be adaptive to each task. Moreover, transforming gradients with meta-learned warp-layers ignores the local geometric features or task-specific knowledge, and may lead to a significant risk of overfitting caused by the increase of parameters. In this paper, we propose ARWarpGrad to guarantee better generalization performance with faster convergence speed by modeling both the cross-task and task-specific knowledge. We introduce Initialization Modulation (IM) to meta-learn to initialize the task-learner specifically. Furthermore, the Mixed Gradient Preprocessing (MGP), which includes the Adaptive Learning Rates (ALR) and the Gaussian Momentum Dropout (GMD), is put forward to provide better adaptive optimization direction and length for task adaptation based on the feature of local geometries. In addition, Memory Regularization (MR) is provided to alleviate the overfitting problem effectively with the use of parameter memory. Ultimately, extensive experiments on three settings demonstrate that ARWarpGrad achieves state-of-the-art performance with convergence acceleration and overfitting prevention characteristics. [ABSTRACT FROM AUTHOR]
- Published
- 2023
- Full Text
- View/download PDF
19. LLP-AAE: Learning from label proportions with adversarial autoencoder.
- Author
-
Wang, Bo, Sun, Yingte, and Tong, Qiang
- Subjects
- *
MACHINE learning , *SUPERVISED learning , *DEEP learning - Abstract
This paper presents an effective weakly supervised learning algorithm LLP-AAE to leverage the adversarial autoencoder (AAE) for learning from label proportions (LLP), in which only the bag-level proportional information is available. Our LLP-AAE utilizes an autoencoder backbone and performs adversarial training in latent space to match the aggregated posterior distribution of hidden coding with the prior distributions. In this way, apart from the reconstruction task, the encoder is also dedicated to producing fake samples, in order to deceive discriminators as far as possible. Ultimately, the encoder is employed as a competent label predictor for unseen data. In addition to the LLP classifier, our model can also achieve controllable samples generation by feeding the decoder with gradually changing latent code, which is proven to be useful for a better LLP performance. We also provide a panoramic explanation for LLP-AAE by regarding the LLP problem as an alternative learning procedure between proportion-based pseudo label generation and discriminative reconstruction. Experiments on six benchmark image datasets demonstrate the advantage of our method both in style manipulation with the latent feature representation and comparable multi-class LLP performance with the state-of-the-art models. [ABSTRACT FROM AUTHOR]
- Published
- 2023
- Full Text
- View/download PDF
20. Land use and land cover classification with hyperspectral data: A comprehensive review of methods, challenges and future directions.
- Author
-
Moharram, Mohammed Abdulmajeed and Sundaram, Divya Meena
- Subjects
- *
ZONING , *GENERATIVE adversarial networks , *CONVOLUTIONAL neural networks , *LAND use , *RECURRENT neural networks , *DROUGHT management - Abstract
Recently, many efforts have been concentrated on land use land cover (LULC) classification due to rapid urbanization, environmental pollution, agriculture drought, frequent floods, and climate change. However, various aspects have attracted hyperspectral imaging due to there being informative discriminative features, such as spectral-spatial features. To this end, this paper is a comprehensive and systematic review of LULC classification using hyperspectral images by reviewing four significant research investigations. Moreover, the four investigations have addressed the following points: (1) the main components of the hyperspectral imaging, the modes of hyperspectral imaging with data acquisition methods, and the intrinsic differences between hyperspectral image and multispectral image, (2) the role of machine learning in LULC classification, and the standard deep learning methods: Convolution Neural Network (CNN), Stacked Autoencoder (SAE), Deep Belief Network (DBN), Recurrent Neural Network (RNN), and Generative Adversarial Network (GAN), (3) the standard benchmark hyperspectral datasets and the evaluation criteria, (4) the main challenges of LULC classification with the possible solutions for the limited training samples issue, the promising future directions, and finally the recent applications for LULC classification. [ABSTRACT FROM AUTHOR]
- Published
- 2023
- Full Text
- View/download PDF
21. Semi-supervised multiple evidence fusion for brain tumor segmentation.
- Author
-
Huang, Ling, Ruan, Su, and Denœux, Thierry
- Subjects
- *
SUPERVISED learning , *DEEP learning , *BRAIN tumors , *MACHINE learning , *DEMPSTER-Shafer theory - Abstract
The performance of deep learning-based methods depends mainly on the availability of large-scale labeled learning data. However, obtaining precisely annotated examples is challenging in the medical domain. Although some semi-supervised deep learning methods have been proposed to train models with fewer labels, only a few studies have focused on the uncertainty caused by the low quality of the images and the lack of annotations. This paper addresses the above issues using Dempster-Shafer theory and deep learning: 1) a semi-supervised learning algorithm is proposed based on an image transformation strategy; 2) a probabilistic deep neural network and an evidential neural network are used in parallel to provide two sources of segmentation evidence; 3) Dempster's rule is used to combine the two pieces of evidence and reach a final segmentation result. Results from a series of experiments on the BraTS2019 brain tumor dataset show that our framework achieves promising results when only some training data are labeled. [ABSTRACT FROM AUTHOR]
- Published
- 2023
- Full Text
- View/download PDF
22. The coming of age of interpretable and explainable machine learning models.
- Author
-
Lisboa, P.J.G., Saralajew, S., Vellido, A., Fernández-Domenech, R., and Villmann, T.
- Subjects
- *
MACHINE learning , *COMING of age , *DATA analysis , *UNIVERSITY research , *JUSTICE administration - Abstract
Machine-learning-based systems are now part of a wide array of real-world applications seamlessly embedded in the social realm. In the wake of this realization, strict legal regulations for these systems are currently being developed, addressing some of the risks they may pose. This is the coming of age of the concepts of interpretability and explainability in machine-learning-based data analysis, which can no longer be seen just as an academic research problem. In this paper, we discuss explainable and interpretable machine learning as post hoc and ante-hoc strategies to address regulatory restrictions and highlight several aspects related to them, including their evaluation and assessment and the legal boundaries of application. [ABSTRACT FROM AUTHOR]
- Published
- 2023
- Full Text
- View/download PDF
23. Distributional reinforcement learning with unconstrained monotonic neural networks.
- Author
-
Théate, Thibaut, Wehenkel, Antoine, Bolland, Adrien, Louppe, Gilles, and Ernst, Damien
- Subjects
- *
REINFORCEMENT learning , *DISTRIBUTION (Probability theory) , *MONOTONIC functions , *CONTINUOUS functions , *ARTIFICIAL intelligence , *MUSCLE weakness - Abstract
• Novel distributional RL algorithm based on unconstrained monotonic neural networks. • Monotonicity ensures the validity of the random return probability distribution. • Methodology to learn different representations of the random return distribution. • Empirical comparison of the probability metrics commonly used in distributional RL. • Critical approximation highlighted for the extensively used Wasserstein distance. The distributional reinforcement learning (RL) approach advocates for representing the complete probability distribution of the random return instead of only modelling its expectation. A distributional RL algorithm may be characterised by two main components, namely the representation of the distribution together with its parameterisation and the probability metric defining the loss. The present research work considers the unconstrained monotonic neural network (UMNN) architecture, a universal approximator of continuous monotonic functions which is particularly well suited for modelling different representations of a distribution. This property enables the efficient decoupling of the effect of the function approximator class from that of the probability metric. The research paper firstly introduces a methodology for learning different representations of the random return distribution (PDF, CDF and QF). Secondly, a novel distributional RL algorithm named unconstrained monotonic deep Q-network (UMDQN) is presented. To the authors' knowledge, it is the first distributional RL method supporting the learning of three , valid and continuous representations of the random return distribution. Lastly, in light of this new algorithm, an empirical comparison is performed between three probability quasi-metrics, namely the Kullback–Leibler divergence, Cramer distance, and Wasserstein distance. The results highlight the main strengths and weaknesses associated with each probability metric together with an important limitation of the Wasserstein distance. [ABSTRACT FROM AUTHOR]
- Published
- 2023
- Full Text
- View/download PDF
24. Introducing multi-dimensional hierarchical classification: Characterization, solving strategies and performance measures.
- Author
-
Montenegro, C., Santana, R., and Lozano, J.A.
- Subjects
- *
HIERARCHICAL Bayes model , *LEARNING strategies , *CLASSIFICATION - Abstract
Classification problems where there exist multiple class variables that need to be jointly predicted are known as Multi-dimensional classification problems. If the labels of these class variables are organized as hierarchies, we can take advantage of specific strategies designed for the Hierarchical classification paradigm. In this paper we present the Multi-dimensional hierarchical classification (MDHC) paradigm, a result of the combination of Multi-dimensional and Hierarchical classification paradigms. We propose four MDHC learning strategies which are designed to exploit the particularities of this new paradigm, combining characteristics of Multi-dimensional and Hierarchical classification strategies. Along with these strategies, we present a framework for classifier comparison in which we use a set of performance measures specifically designed for MDHC, and a procedure to create MDHC synthetic scenarios. Using this framework and the performance measures presented, we study how characteristics of the MDHC problems influence the performance of the different MDHC strategies proposed, and compare them to other non-MDHC strategies. [ABSTRACT FROM AUTHOR]
- Published
- 2023
- Full Text
- View/download PDF
25. An active memristor based rate-coded spiking neural network.
- Author
-
Amin Fida, Aabid, Khanday, Farooq A., and Mittal, Sparsh
- Subjects
- *
BIOLOGICAL neural networks , *MACHINE learning , *MEMRISTORS , *BOOLEAN functions , *ONLINE education - Abstract
• Physical behaviors of memristive systems can be related to the bio-physical dynamics of biological neural elements. • Spiking behaviors subject to input stimuli of LIF neurons made of memristive elements can be extrapolated to develop on-chip learning algorithms. • Rate coding is a viable alternative to temporal or population coding for in-hardware SNNs. • It is possible to perform a non-linear functions like XOR using a single neuron in SNNs. • A hybrid approach relying on ANN like gradient calculation can be used to learn in SNNs. Neuromorphic computing is a novel computing paradigm that aims to mimic the behavior of biological neural networks for efficiently solving complex problems. While CMOS based neurons and synapses have been developed, they are limited in their ability to demonstrate bio-realistic dynamics. This, coupled with the fact that a huge number of these individual devices are required to build neurons and synapses, limits the scaling and power efficiency of such systems. A viable answer to this problem is neuromemristive systems that are based on memristor devices. These devices exhibit physical behaviors that can be related to the bio-physical dynamics of synapses and neurons. In this paper, a rate-coded all memristive "spiking neural network" (SNN) is presented. The proposed SNN is built with an active memristor neuron based on vanadium dioxide (VO 2) coupled with a non-volatile memristor synapse. The results are validated by first simulating spiking versions of two Boolean functions viz., AND and XOR gates in SPICE. With features extracted from the small neural nets, a large-scale 3-layer spiking neural network is then simulated in Python which yields a validation accuracy of 87% on the MNIST dataset of handwritten digits. One of the prime features of this work is the realization of the XOR function using a single neuron which is not possible without the use of 2-layers of neurons in traditional neural networks. Another significant contribution is the utilization of a gradient-based learning approach for online training of a large-scale SNN. For this, we use the inherent activation function (Sigmoid/ReLU) of the proposed neuron design. [ABSTRACT FROM AUTHOR]
- Published
- 2023
- Full Text
- View/download PDF
26. Graph learning for latent-variable Gaussian graphical models under laplacian constraints.
- Author
-
Li, Ran, Lin, Jiming, Qiu, Hongbing, Zhang, Wenhui, and Wang, Junyi
- Subjects
- *
GRAPH theory , *LAPLACIAN matrices , *SPECTRAL theory , *LATENT variables , *SPARSE graphs , *SIGNAL processing , *MACHINE learning - Abstract
• The problem of graph Laplacian estimation with latent variables is formulated. • An multi-block ADMM algorithm is proposed to solve the problem. • The proposed method can estimate the conditional correlation of observed variables. In recent years, graph learning for smooth signals under Laplacian constraints has attracted increasing attention due to the wide application of graph Laplacian matrix in spectral graph theory, machine learning, and graph signal processing tasks. Standard graph learning methods usually assume that graphs are sparse, but the correlation between real-world entities is only sometimes sparse because of some common and potential effects. In this paper, we model these common effects as latent variables and assume that the Gaussian graphical model (GGM) under Laplacian constraints is conditionally sparse given latent variables but marginally non-sparse. Based on this assumption, the graph learning problem is formulated in a regularized maximum marginal likelihood (MML) framework with a sparse plus low-rank decomposition form. The specialized algorithm is developed to solve the proposed graph learning problem by incorporating Laplacian constraints into a multi-block alternating direction method of multipliers (ADMM) with proximal regularization terms. The experiments conducted on synthetic and real-world data sets demonstrate that the proposed graph learning method outperforms the standard method in inferring the sparsity pattern of the conditional graphical model of observed variables with the presence of latent variables. [ABSTRACT FROM AUTHOR]
- Published
- 2023
- Full Text
- View/download PDF
27. Simulation-based evaluation of model-free reinforcement learning algorithms for quadcopter attitude control and trajectory tracking.
- Author
-
Caffyn Yuste, Pablo, Iglesias Martínez, José Antonio, and Sanchis de Miguel, María Araceli
- Subjects
- *
MACHINE learning , *REINFORCEMENT learning , *SELECTION (Plant breeding) , *ALGORITHMS , *SEEDS - Abstract
General use quadcopters have been under development for over a decade but many of their potential applications are still under evaluation and have not yet been adopted in many of the areas that could benefit from their use. While the current generation of quadcopters use a mature set of control algorithms, the next steps, especially as autonomous features are developed, should involve a more complex learning capability to be able to adapt to unknown circumstances in a safe and reliable way. This paper provides baseline quadcopter control models learnt using eight general reinforcement learning (RL) algorithms in a simulated environment, with the object of establishing a reference performance, both in terms of precision and generation cost, for a simple set of trajectories. Each algorithm uses a tailored set of hyperparameters while, additionally, the influence of random seeds is also studied. While not all algorithms converge in the allocated computing budget, the more complex ones are able to provide stable and precise control models. This paper recommends the use of the TD3 algorithm as a reference for comparison with new RL algorithms. Additional guidance for future work is provided based on the weaknesses identified in the learning process, especially regarding the strong dependence of agent performance on random seeds. • Quadcopter control models learnt using model-free reinforcement learning algorithms. • Trained and tested in a simulated environment using OpenAI gym and pybullet. • Trained using small displacements, tested on trajectories unknown to the agent. • TD3 algorithm shows the most consistent performance. • All algorithms show strong dependence on the random seed selection. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
28. Toward cross-subject and cross-session generalization in EEG-based emotion recognition: Systematic review, taxonomy, and methods.
- Author
-
Apicella, Andrea, Arpaia, Pasquale, D'Errico, Giovanni, Marocco, Davide, Mastrati, Giovanna, Moccaldi, Nicola, and Prevete, Roberto
- Subjects
- *
EMOTION recognition , *FEATURE extraction , *GENERALIZATION , *DATABASE searching , *ELECTROENCEPHALOGRAPHY - Abstract
A systematic review on machine-learning strategies for improving generalization in electroencephalography-based emotion classification was realized. In particular, cross-subject and cross-session generalization was focused. In this context, the non-stationarity of electroencephalographic (EEG) signals is a critical issue and can lead to the Dataset Shift problem. Several architectures and methods have been proposed to address this issue, mainly based on transfer learning methods. In this review, 449 papers were retrieved from the Scopus , IEEE Xplore and PubMed databases through a search query focusing on modern machine learning techniques for generalization in EEG-based emotion assessment. Among these papers, 79 were found eligible based on their relevance to the problem. Studies lacking a specific cross-subject or cross-session validation strategy, or making use of other biosignals as support were excluded. On the basis of the selected papers' analysis, a taxonomy of the studies employing Machine Learning (ML) methods was proposed, together with a brief discussion of the different ML approaches involved. The studies reporting the best results in terms of average classification accuracy were identified, supporting that transfer learning methods seem to perform better than other approaches. A discussion is proposed on the impact of (i) the emotion theoretical models and (ii) psychological screening of the experimental sample on the classifier performances. [Display omitted] • The non-stationarity of EEG signals can lead to the Dataset Shift problem. • Transfer learning methods improve generalizability in EEG-based emotion classification. • Adaptive feature extraction also in combination with transfer learning are promising for generalization. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
29. Differentially private stochastic gradient descent with low-noise.
- Author
-
Wang, Puyu, Lei, Yunwen, Ying, Yiming, and Zhou, Ding-Xuan
- Subjects
- *
MACHINE learning , *CONVEX sets , *PRIVACY - Abstract
Modern machine learning algorithms aim to extract fine-grained information from data to provide accurate predictions, which often conflicts with the goal of privacy protection. This paper addresses the practical and theoretical importance of developing privacy-preserving machine learning algorithms that ensure good performance while preserving privacy. In this paper, we focus on the privacy and utility (measured by excess risk bounds) performances of differentially private stochastic gradient descent (SGD) algorithms in the setting of stochastic convex optimization. Specifically, we examine the pointwise problem in the low-noise setting for which we derive sharper excess risk bounds for the differentially private SGD algorithm. In the pairwise learning setting, we propose a simple differentially private SGD algorithm based on gradient perturbation. Furthermore, we develop novel utility bounds for the proposed algorithm, proving that it achieves optimal excess risk rates even for non-smooth losses. Notably, we establish fast learning rates for privacy-preserving pairwise learning under the low-noise condition, which is the first of its kind. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
30. Progressive expansion: Cost-efficient medical image analysis model with reversed once-for-all network training paradigm.
- Author
-
Lim, Shin Wei, Chan, Chee Seng, Mohd Faizal, Erma Rahayu, and Ewe, Kok Howg
- Subjects
- *
COMPUTER-assisted image analysis (Medicine) , *IMAGE analysis , *DIAGNOSTIC imaging , *ARTIFICIAL intelligence , *IMAGE segmentation , *HIPPOCAMPUS (Brain) - Abstract
Low computational cost artificial intelligence (AI) models are vital in promoting the accessibility of real-time medical services in underdeveloped areas. The recent Once-For-All (OFA) network (without retraining) can directly produce a set of sub-network designs with Progressive Shrinking (PS) algorithm; however, the training resource and time inefficiency downfalls are apparent in this method. In this paper, we propose a new OFA training algorithm, namely the Progressive Expansion (ProX) to train the medical image analysis model. It is a reversed paradigm to PS, where technically we train the OFA network from the minimum configuration and gradually expand the training to support larger configurations. Empirical results showed that the proposed paradigm could reduce training time up to 68%; while still being able to produce sub-networks that have either similar or better accuracy compared to those trained with OFA-PS on ROCT (classification), BRATS and Hippocampus (3D-segmentation) public medical datasets. The code implementation for this paper is accessible at: https://github.com/shin-wl/ProX-OFA. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
31. An efficient multi-metric learning method by partitioning the metric space.
- Author
-
Yuan, Chao and Yang, Liming
- Subjects
- *
SUPERVISED learning , *MACHINE learning , *LOGARITHMIC functions , *PATTERN recognition systems , *LEARNING - Abstract
Metric learning has attracted significant attention due to its high effectiveness and efficiency for pattern recognition task. Traditional supervised metric learning algorithms attempt to seek a global distance metric with labeled samples. When data are represented with multimodal and only limited supervision information is available, these approaches are insufficient to obtain satisfactory results. In this paper, we develop a robust semi-supervised multi-metric learning method (RSMM) to improve classification performance. The proposed RSMM learns multiple local metrics and a background metric instead of a single global metric. Specifically, we divide the metric space into influential regions and background region, and then regulate the effectiveness of each local metric to be within the related regions. Simultaneously, a geometrically interpretable, symmetric distance is defined with local metrics and background metric. Based on the resultant learning bounds, we obtain the regularization term to improve the classifier's generalization ability. Moreover, the manifold regularization term is introduced to preserve the supervision information as well as geometry structure. The substantial unlabeled samples may cause potential threats and large uncertainties, so the logarithmic loss function is utilized to enhance the robustness. An efficient gradient descent algorithm is exploited to solve the non-convex challenging problem. To further understand the proposed algorithm, we theoretically derive its robustness and generalization error bounds. Finally, numerical experiments on UCI datasets and image datasets demonstrate the feasibility and validity of the RSMM. [ABSTRACT FROM AUTHOR]
- Published
- 2023
- Full Text
- View/download PDF
32. Towards an ML-based semantic IoT for pandemic management: A survey of enabling technologies for COVID-19.
- Author
-
Zgheib, Rita, Chahbandarian, Ghazar, Kamalov, Firuz, Messiry, Haythem El, and Al-Gindy, Ahmed
- Subjects
- *
COVID-19 pandemic , *MACHINE learning , *PANDEMICS , *COVID-19 , *ARTIFICIAL intelligence , *DIGITAL technology - Abstract
The connection between humans and digital technologies has been documented extensively in the past decades but needs to be evaluated through the current global pandemic. Artificial Intelligence(AI), with its two strands, Machine Learning (ML) and Semantic Reasoning, has proven to be a great solution to provide efficient ways to prevent, diagnose and limit the spread of COVID-19. IoT solutions have been widely proposed for COVID-19 disease monitoring, infection geolocation, and social applications. In this paper, we investigate the usage of the three technologies for handling the COVID-19 pandemic. For this purpose, we surveyed the existing ML applications and algorithms proposed during the pandemic to detect COVID-19 disease using symptom factors and image processing. The survey includes existing approaches including semantic technologies and IoT systems for COVID-19. Based on the survey result, we classified the main challenges and the solutions that could solve them. The study proposes a conceptual framework for pandemic management and discusses challenges and trends for future research. [ABSTRACT FROM AUTHOR]
- Published
- 2023
- Full Text
- View/download PDF
33. Neural collapse inspired attraction–repulsion-balanced loss for imbalanced learning.
- Author
-
Xie, Liang, Yang, Yibo, Cai, Deng, and He, Xiaofei
- Subjects
- *
DEEP learning , *MATHEMATICAL optimization , *LEARNING , *IMAGE segmentation - Abstract
Class imbalance distribution widely exists in real-world engineering. However, the mainstream optimization algorithms that seek to minimize error will trap the deep learning model in sub-optimums when facing extreme class imbalance. It seriously harms the classification precision, especially in the minor classes. The essential reason is that the gradients of the classifier weights are imbalanced among the components from different classes. In this paper, we propose Attraction–Repulsion-Balanced Loss (ARB-Loss) to balance the different components of the gradients. We perform experiments on large-scale classification and segmentation datasets, and our ARB-Loss can achieve state-of-the-art performance via only one-stage training instead of 2-stage learning like nowadays SOTA works. [ABSTRACT FROM AUTHOR]
- Published
- 2023
- Full Text
- View/download PDF
34. Cost-sensitive learning with modified Stein loss function.
- Author
-
Fu, Saiji, Tian, Yingjie, Tang, Jingjing, and Liu, Xiaohui
- Subjects
- *
MACHINE learning , *LEARNING - Abstract
Cost-sensitive learning (CSL), which has gained widespread attention in class imbalance learning (CIL), can be implemented either by tuning penalty parameters or by designing new loss functions. In this paper, we propose a cost-sensitive learning method with a modified Stein loss function (CSMS) and a robust CSMS (RCSMS). Specifically, CSMS is flexible, as it realizes CSL from above two aspects simultaneously. In contrast, RCSMS merely achieves CSL by tuning penalty parameters, but the adopted loss function makes it insensitive to noise. To our best knowledge, it is the first time for Stein loss function derived from statistics to be applied in machine learning, which not only offers two alternative class imbalance solutions but also provides a novel idea for the design of loss functions in CIL. The mini-batch stochastic sub-gradient descent (MBGD) approach is employed to optimize CSMS and RCSMS. Meanwhile, the Rademacher complexity is used to analyze their generalization error bounds. Extensive experiments profoundly confirm the superiority of both models over benchmarks. [ABSTRACT FROM AUTHOR]
- Published
- 2023
- Full Text
- View/download PDF
35. Kernel reconstruction learning.
- Author
-
Wu, Yun and Xiong, Shifeng
- Subjects
- *
PROBABILITY density function , *LOGISTIC regression analysis , *SUPPORT vector machines , *LEARNING problems , *INTERPOLATION algorithms , *MACHINE learning - Abstract
• We propose a class of kernel interpolation-based methods, called kernel reconstruction learning, for solving machine learning problems. • We prove a reconstruction representer theorem, which indicates that conventional kernel methods can be viewed as special cases of kernel reconstruction learning. • We propose the kernel reconstruction vector machine, kernel reconstruction logistic regression, and kernel reconstruction density estimation methods, and show that they outperform popular kernel methods. This paper proposes a class of kernel interpolation-based methods, called kernel reconstruction learning, for solving machine learning problems. Kernel reconstruction learning uses kernel interpolators to reconstruct the unknown functions, which are needed to estimate in the problem, with estimated function values at selected knots. It can be applied to any learning problem that involves function estimation. We prove a reconstruction representer theorem, which indicates that conventional kernel methods, including kernel ridge regression, kernel support vector machine, and kernel logistic regression, can be viewed as special cases of kernel reconstruction learning. Furthermore, kernel reconstruction learning provides new algorithms for large datasets. The kernel reconstruction vector machine, kernel reconstruction logistic regression, and kernel reconstruction density estimation are discussed in detail. With appropriate implementations, they are shown to have higher prediction/estimation accuracy and/or less computational cost than popular kernel methods. [ABSTRACT FROM AUTHOR]
- Published
- 2023
- Full Text
- View/download PDF
36. problexity—An open-source Python library for supervised learning problem complexity assessment.
- Author
-
Komorniczak, Joanna and Ksieniewicz, Paweł
- Subjects
- *
SUPERVISED learning , *PYTHON programming language , *MACHINE learning , *COMMUNITIES , *LEARNING communities , *C++ - Abstract
The problem's complexity assessment is an essential element of many topics in the supervised learning domain. It plays a significant role in meta-learning – becoming the basis for determining meta-attributes or multi-criteria optimization – allowing the evaluation of the training set resampling without needing to rebuild the recognition model. The tools currently available for the academic community, which would enable the calculation of problem complexity measures, are available only as libraries of the C++ and R languages. This paper describes the software module that allows for the estimation of 22 classification complexity measures and 12 regression complexity measures for the Python language – compatible with the scikit-learn programming interface – allowing for the implementation of research using them in the most popular programming environment of the machine learning community. [ABSTRACT FROM AUTHOR]
- Published
- 2023
- Full Text
- View/download PDF
37. Considering three elements of aesthetics: Multi-task self-supervised feature learning for image style classification.
- Author
-
Zhang, Hua, Luo, Yizhang, Zhang, Lingjun, Wu, Yifan, Wang, Muwei, and Shen, Zhuonan
- Subjects
- *
SUPERVISED learning , *COGNITIVE styles , *MACHINE learning , *AESTHETICS - Abstract
• Applying Self-Supervised Learning to the Learning of Unlabeled Aesthetic Image Features. • The current problem of self-supervised learning is adapted based on the particularity of aesthetic images. • Multiple self-supervised learning tasks are designed based on the three elements of aesthetics. • A style feature learning method based on multi-task self-supervised learning is proposed. Image style classification is the basis of computational aesthetics, and its role has become increasingly important with the rise of computational aesthetics. Most of the current image style classification methods use supervised learning for model training. These methods require a large number of expensive aesthetic style labels. Unlike existing methods, self-supervised learning can perform feature learning on many current unlabeled style images, thereby alleviating the constraint that current supervised learning methods require a large amount of labeled data. However, the self-supervised learning method also poses the problem that it is difficult to fully characterize the highly subjective aesthetic style characteristics. Therefore, this study proposes a multi-task self-supervised style feature learning algorithm considering the three elements of aesthetics. The three elements of aesthetics include compositional rules, luminance and color. The algorithm designs multiple self-supervised learning tasks from multiple perspectives and proposes a joint learning method for multiple self-supervised learning tasks, so that the model can learn the style features of images more comprehensively than ordinary self-supervised learning methods. The experimental results on three large image style datasets also show that the method proposed in this paper can effectively learn the style features of images and outperform most style feature learning algorithms based on supervised learning in terms of classification results. [ABSTRACT FROM AUTHOR]
- Published
- 2023
- Full Text
- View/download PDF
38. What-Where-When Attention Network for video-based person re-identification.
- Author
-
Zhang, Chenrui, Chen, Ping, Lei, Tao, Wu, Yangxu, and Meng, Hongying
- Subjects
- *
VIDEO surveillance , *MACHINE learning , *TIME-varying networks - Abstract
Video-based person re-identification plays a critical role in intelligent video surveillance by learning temporal correlations from consecutive video frames. Most existing methods aim to solve the challenging variations of pose, occlusion, backgrounds and so on by using attention mechanism. They almost all draw attention to the occlusion and learn occlusion-invariant video representations by abandoning the occluded area or frames, while the other areas in these frames contain sufficient spatial information and temporal cues. To overcome these drawbacks, this paper proposes a comprehensive attention mechanism covering what , where , and when to pay attention in the discriminative spatial-temporal feature learning, namely What-Where-When Attention Network (W3AN). Concretely, W3AN designs a spatial attention module to focus on pedestrian identity and obvious attributes by the importance estimating layer (What and Where), and a temporal attention module to calculate the frame-level importance (when), which is embedded into a graph attention network to exploit temporal attention features rather than computing weighted average feature for video frames like existing methods. Moreover, the experiments on three widely-recognized datasets demonstrate the effectiveness of our proposed W3AN model and the discussion of major modules elaborates the contributions of this paper. [ABSTRACT FROM AUTHOR]
- Published
- 2022
- Full Text
- View/download PDF
39. A method of traffic police detection based on attention mechanism in natural scene.
- Author
-
Zheng, Ying, Bao, Hong, Meng, Chaochao, and Ma, Nan
- Subjects
- *
TRAFFIC police , *TRAFFIC monitoring , *POLICE training , *DRIVERLESS cars , *FRAMES (Social sciences) , *MACHINE learning - Abstract
The complex and varied urban road conditions have always been a difficult and a pivotal component in the study of driverless technology especially at intersections. In China, it is necessary for driverless cars to understand the gestures of traffic police. To identify the traffic police gesture at the intersection, the key step is to detect the traffic police at the intersection. At present, the research on traffic police detection is still in its infancy, there exists common problems such as slow detection speed and other real time problems in this method, and there is not a standardized traffic police data set ether. For the real time problems, this paper introduces the attention mechanism, and proposes a new real-time detection method of traffic police based on attention mechanism. The method proposed in this paper has strong robustness and can quickly complete the target detection task. For the data set problem, this paper analyzes and discloses similar data sets published through research. In the meanwhile, this paper collects 24,530 video data on the actual road, and it extracts and saves 12,000 pictures containing traffic police at frame rate of 30FPS as traffic police detection training and validated data set. [ABSTRACT FROM AUTHOR]
- Published
- 2021
- Full Text
- View/download PDF
40. Convex formulation for multi-task L1-, L2-, and LS-SVMs.
- Author
-
Ruiz, Carlos, Alaíz, Carlos M., and Dorronsoro, José R.
- Subjects
- *
ERROR functions , *SUPPORT vector machines , *MACHINE learning - Abstract
Quite often a machine learning problem lends itself to be split in several well-defined subproblems, or tasks. The goal of Multi-Task Learning (MTL) is to leverage the joint learning of the problem from two different perspectives: on the one hand, a single, overall model, and on the other hand task-specific models. In this way, the found solution by MTL may be better than those of either the common or the task-specific models. Starting with the work of Evgeniou et al., support vector machines (SVMs) have lent themselves naturally to this approach. This paper proposes a convex formulation of MTL for the L1-, L2- and LS-SVM models that results in dual problems quite similar to the single-task ones, but with multi-task kernels; in turn, this makes possible to train the convex MTL models using standard solvers. As an alternative approach, the direct optimal combination of the already trained common and task-specific models can also be considered. In this paper, a procedure to compute the optimal combining parameter with respect to four different error functions is derived. As shown experimentally, the proposed convex MTL approach performs generally better than the alternative optimal convex combination, and both of them are better than the straight use of either common or task-specific models. [ABSTRACT FROM AUTHOR]
- Published
- 2021
- Full Text
- View/download PDF
41. Uncertainty quantification in extreme learning machine: Analytical developments, variance estimates and confidence intervals.
- Author
-
Guignard, Fabian, Amato, Federico, and Kanevski, Mikhail
- Subjects
- *
MACHINE learning , *UNCERTAINTY , *PYTHON programming language , *HETEROSCEDASTICITY , *CONFIDENCE intervals - Abstract
• Analytical developments support the understanding of ELM variability. • A special attention is paid to the impact of the input weight randomness. • Variance estimates for homoskedastic, heteroskedastic, regularized cases are given. • The possibility of constructing accurate confidence intervals is discussed. • A new Python library allows the computation of the proposed estimates. Uncertainty quantification is crucial to assess prediction quality of a machine learning model. In the case of Extreme Learning Machines (ELM), most methods proposed in the literature make strong assumptions on the data, ignore the randomness of input weights or neglect the bias contribution in confidence interval estimations. This paper presents novel estimations that overcome these constraints and improve the understanding of ELM variability. Analytical derivations are provided under general assumptions, supporting the identification and the interpretation of the contribution of different variability sources. Under both homoskedasticity and heteroskedasticity, several variance estimates are proposed, investigated, and numerically tested, showing their effectiveness in replicating the expected variance behaviours. Finally, the feasibility of confidence intervals estimation is discussed by adopting a critical approach, hence raising the awareness of ELM users concerning some of their pitfalls. The paper is accompanied with a scikit-learn compatible Python library enabling efficient computation of all estimates discussed herein. [ABSTRACT FROM AUTHOR]
- Published
- 2021
- Full Text
- View/download PDF
42. Intrusion detection approach based on optimised artificial neural network.
- Author
-
Choraś, Michał and Pawlicki, Marek
- Subjects
- *
ARTIFICIAL neural networks , *MACHINE learning , *BIOLOGICALLY inspired computing - Abstract
Intrusion Detection, the ability to detect malware and other attacks, is a crucial aspect to ensure cybersecurity. So is the ability to identify this myriad of attacks. Artificial Neural Networks (as well as other machine learning bio-inspired approaches) are an established and proven method of accurate classification. ANNs are extremely versatile – a wide range of setups can achieve significantly different classification results. The main objective and contribution of this paper is the evaluation of the way the hyperparameters can influence the final classification result. In this paper, a wide range of ANN setups is put to comparison. We have performed our experiments on two benchmark datasets, namely NSL-KDD and CICIDS2017. The most effective arrangement achieves the multi-class classification accuracy of 99.909% on an established benchmark dataset. [ABSTRACT FROM AUTHOR]
- Published
- 2021
- Full Text
- View/download PDF
43. MAENet: A novel multi-head association attention enhancement network for completing intra-modal interaction in image captioning.
- Author
-
Hu, Nannan, Fan, Chunxiao, Ming, Yue, and Feng, Fan
- Subjects
- *
COMPUTER vision , *HEAD , *MACHINE learning - Abstract
Image captioning attracts much attention as it bridges computer vision and natural language processing. Recent works show that transformer-based models with the multi-head self-attention can explore intra-modal interactions for generating high-quality image captions. However, the subspace of each attention head is operated independently in these multi-head attention methods, which ignores the association between attention heads and makes the learning of intra-modal interaction incomplete. In this paper, we propose a Multi-head Association Attention Enhancement Network (MAENet) for image captioning, which leverages a novel Multi-head Association Attention Enhancement (MAE) block for completing intra-modal interaction learning. The proposed MAE block contains Multi-head Association Attention (MAA) and Attention Enhancement (AE) module.The MAA calculates the contributive weight of different attention heads, and captures the associated information from adjacent attention subspaces via learned associative parameters. The AE module follows with the MAA to further enhance the association attention results through an additional spatial and channel-wise attention aggregation. It's worth noting that the MAE block is a plug-and-play module that can be cascaded with other multi-head attention mechanisms. Extensive experiments on MS COCO show that our model achieves a quite competitive performance, especially for the model of MAE block cascaded with X-linear attention obtains the best-reported SPICE performance of 23.5 % on the Karpathy test split. This clearly demonstrates that the proposed model can better model the interactive information and result in superior captions. [ABSTRACT FROM AUTHOR]
- Published
- 2023
- Full Text
- View/download PDF
44. MUSE: Multi-faceted attention for signed network embedding.
- Author
-
Yan, Dengcheng, Zhang, Youwen, Xie, Wenxin, Jin, Ying, and Zhang, Yiwen
- Subjects
- *
DATA mining , *MACHINE learning - Abstract
Signed network embedding is an approach to learning low-dimensional representations of nodes in signed networks with both positive and negative links, which facilitates downstream tasks such as link prediction with general data mining frameworks. Due to the distinct properties and significant added value of negative links, existing signed network embedding methods usually design dedicated methods based on social theories such as balance theory and status theory. However, existing signed network embedding methods ignore the characteristics of multiple facets of each node and mix them up in one single representation, which limits the ability to capture the fine-grained attentions between node pairs. In this paper, we propose MUSE , a MU lti-faceted attention-based S igned network E mbedding framework to tackle this problem. Specifically, a joint intra- and inter-facet attention mechanism is introduced to aggregate fine-grained information from neighbor nodes. Moreover, balance theory is also utilized to guide information aggregation from multi-order balanced and unbalanced neighbors. Experimental results on four real-world signed network datasets demonstrate the effectiveness of our proposed framework. [ABSTRACT FROM AUTHOR]
- Published
- 2023
- Full Text
- View/download PDF
45. Unsupervised video summarization using deep Non-Local video summarization networks.
- Author
-
Zang, Sha-Sha, Yu, Hui, Song, Yan, and Zeng, Ru
- Subjects
- *
RECURRENT neural networks , *VIDEO summarization , *REINFORCEMENT learning , *MACHINE learning , *STATISTICAL decision making - Abstract
Video summarization is to extract effective information from videos to quickly obtain the most informative summary. Most of the existing video summarization methods use recurrent neural networks and their variants such as long and short-term memory (LSTM), to simulates the variable range time dependence between video frames. However, those methods can only process serial inputs of the video frames along with the hidden layer information from the previous time step, which affects the performance and the quality of video summarization. To tackle this issue, we present a deep non-local video summarization network (DN-VSN) for original video abstracts in this paper. Our unsupervised model treats video summarization as a sequence of decision problems. Given an input video, the probability that a video frame is selected as a part of the summary is obtained through a non-local convolutional network, and a strategy gradient algorithm of reinforcement learning is adopted for optimization in the training phase. The proposed method has been tested on four widely used datasets. The experimental results show the superiority of the proposed unsupervised model over the state-of-the-art methods. [ABSTRACT FROM AUTHOR]
- Published
- 2023
- Full Text
- View/download PDF
46. AI meets UAVs: A survey on AI empowered UAV perception systems for precision agriculture.
- Author
-
Su, Jinya, Zhu, Xiaoyong, Li, Shihua, and Chen, Wen-Hua
- Subjects
- *
DEEP learning , *MACHINE learning , *PRECISION farming , *ARTIFICIAL intelligence , *SELF-efficacy , *GRAPHICS processing units - Abstract
Precision Agriculture (PA) promises to boost crop productivity while reducing agricultural costs and environmental footprints, and therefore is attracting ever-increasing interests in both academia and industry. This management strategy is underpinned by various advanced technologies including Unmanned Aerial Vehicle (UAV) sensing systems and Artificial Intelligence (AI) perception algorithms. In particular, due to their unique advantages such as a low cost, high spatio-temporal resolutions, flexibility, automation functions and minimized risk of operation, UAV sensing systems have been extensively applied in many civilian applications including PA since 2010. In parallel, AI algorithms (deep learning since 2012 in particular) are also drawing ever-increasing attention in different fields, since they are able to analyse an unprecedented volume/velocity/variety of data (semi-) automatically, which are also becoming computationally practical with the advancements of cloud computing, Graphics Processing Units and parallel computing. In this survey paper, therefore, a thorough review is performed on recent use of UAV sensing systems (e.g., UAV platforms, external sensing units) and AI algorithms (mainly supervised learning algorithms) in PA applications throughout the crop life-cycle, as well as the challenges and prospects for future development of UAVs and AI in agriculture sector. It is envisioned that this review is able to provide a timely technical reference, demystifying and promoting research, deployment and successful exploitation of AI empowered UAV perception systems for PA, and therefore contributing to addressing future agricultural and human nutrition challenges. [ABSTRACT FROM AUTHOR]
- Published
- 2023
- Full Text
- View/download PDF
47. Low-light image enhancement with knowledge distillation.
- Author
-
Li, Ziwen, Wang, Yuehuan, and Zhang, Jinpu
- Subjects
- *
IMAGE intensifiers , *MACHINE learning , *CONVOLUTIONAL neural networks - Abstract
Low-light image enhancement studies how to improve the quality of images captured under poor lighting conditions, which is of real-world importance. Currently, convolutional neural network (CNN)-based methods with state-of-the-art performance have become the mainstream of research. However, most CNN-based methods improve the performance of the algorithm by increasing the width and depth of the neural network, which requires large computing device resources. In this paper, we propose a knowledge distillation method for low light image enhancement. The proposed method uses a teacher-student framework in which the teacher network tries to transfer the rich knowledge to the student network. The student network learns the knowledge of image enhancement under the supervision of ground truth images and under the guidance of the teacher network simultaneously. Knowledge transfer between the teacher-student network is accomplished by distillation loss based on attention maps. We designed a gradient-guided low-light image enhancement network that can be divided into an enhancement branch and a gradient branch, where the enhancement branch is learned under the guidance of the gradient branch to better preserve structural information. The teacher and student networks use a similar structure, but they have different model sizes. The teacher network has more parameters and more powerful learning capabilities than the student network. With the help of knowledge distillation, our approach can improve the performance of the student network without increasing the computational burden during the testing phase. The qualitative and quantitative experimental results demonstrate the superiority of our method compared to the state-of-the-art methods. [ABSTRACT FROM AUTHOR]
- Published
- 2023
- Full Text
- View/download PDF
48. Recurrent attention unit: A new gated recurrent unit for long-term memory of important parts in sequential data.
- Author
-
Niu, Zhaoyang, Zhong, Guoqiang, Yue, Guohua, Wang, Li-Na, Yu, Hui, Ling, Xiao, and Dong, Junyu
- Subjects
- *
LONG-term memory , *RECURRENT neural networks , *NATURAL language processing , *INTERIOR-point methods , *MACHINE learning - Abstract
Gated recurrent unit (GRU) is a variant of the recurrent neural network (RNN). It has been widely used in many applications, such as handwriting recognition and natural language processing. However, GRU can only memorize the sequential information, but lacks the capability of adaptively paying attention to important parts in the sequences. In this paper, we propose a novel RNN model, called recurrent attention unit (RAU), which can seamlessly integrate the attention mechanism into the interior of the GRU cell by adding an attention gate. The attention gate enhances the ability of RAU to remember long-term information and pay attention to important parts in the sequential data. Extensive experiments on adding problem, image classification, sentiment classification and language modeling show that RAU consistently outperforms GRU and other related models. [ABSTRACT FROM AUTHOR]
- Published
- 2023
- Full Text
- View/download PDF
49. Robust projection twin extreme learning machines with capped [formula omitted]-norm distance metric.
- Author
-
Yang, Yang, Xue, Zhenxia, Ma, Jun, and Chang, Xia
- Subjects
- *
MACHINE learning , *KERNEL functions - Abstract
• A novel classification method named projection twin extreme learning machine (PTELM) is proposed. • A new method termed as capped L1-norm projection twin extreme learning machine (CL1-PTELM) is proposed by introducing capped L1-norm distance metric into PTELM. • Two efficient algorithms are designed for our PTELM and CL1-PTELM, and two theorems are proved for CL1-PTELM's convergence and local optimality. • Sufficient experiments on multiple datasets demonstrate the effectiveness of the proposed methods. In this paper, we incorporate the idea of projection twin support vector machines (PTSVM) into the basic framework of twin extreme learning machines (TELM) and first propose a novel binary classifier named projection twin extreme learning machines (PTELM). PTELM is to seek two projection directions in the TELM feature space, such that the projected samples of one class are well separated from those of the other class. Compared with the PTSVM, PTELM tackles nonlinear cases without using several fixed kernel functions, thus PTELM is less sensitive to use specified parameters and can get better classification accuracy. Then, a new capped L 1 -norm PTELM (C L 1 -PTELM) is proposed by introducing capped L 1 -norm distance metric in PTELM to reduce the effect of outliers. C L 1 -PTELM overcomes the disadvantages of L 2 -norm distance metric and hinge loss. Thus, C L 1 -PTELM enhances the robust performance of our PTELM. Finally, two effective algorithms are designed to solve the problem of PTELM and to deal with the challenge of C L 1 -PTELM brought by non-convex optimization problem, respectively. Simultaneously, we theoretically prove the convergence and local optimality of C L 1 -PTELM algorithm. Numerical experiments on three synthetic datasets and several UCI datasets show the feasibility and effectiveness of our proposed methods. [ABSTRACT FROM AUTHOR]
- Published
- 2023
- Full Text
- View/download PDF
50. Multi-objective fuzzy Q-learning to solve continuous state-action problems.
- Author
-
Asgharnia, Amirhossein, Schwartz, Howard, and Atia, Mohamed
- Subjects
- *
MACHINE learning , *REINFORCEMENT learning , *FUZZY logic , *MATHEMATICAL optimization , *FUZZY systems , *SOCIAL problems - Abstract
Many real world problems are multi-objective. Thus, the need for multi-objective learning and optimization algorithms is inevitable. Although the multi-objective optimization algorithms are well-studied, the multi-objective learning algorithms have attracted less attention. In this paper, a fuzzy multi-objective reinforcement learning algorithm is proposed, and we refer to it as the multi-objective fuzzy Q-learning (MOFQL) algorithm. The algorithm is implemented to solve a bi-objective reach-avoid game. The majority of the multi-objective reinforcement algorithms proposed address solving problems in the discrete state-action domain. However, the MOFQL algorithm can also handle problems in a continuous state-action domain. A fuzzy inference system (FIS) is implemented to estimate the value function for the bi-objective problem. We used a temporal difference (TD) approach to update the fuzzy rules. The proposed method is a multi-policy multi-objective algorithm and can find the non-convex regions of the Pareto front. [ABSTRACT FROM AUTHOR]
- Published
- 2023
- Full Text
- View/download PDF
Discovery Service for Jio Institute Digital Library
For full access to our library's resources, please sign in.