Search

This paper introduces the Ulimisana Optimisation Algorithm enabled Population Based Training (PBT-UOA) framework which allows for hyperparameters to be fine-tuned using a population based meta-heuristic algorithm at the same time as parameters are being optimised. Models are trained until near-convergence on the updated hyperparameters and the parameters of the best performing model are shared to warm start the other models in the next hyperparameter tuning iteration. In the PBT-UOA, all models are trained using the same dataset. This framework performed better than the Bayesian Optimisation algorithm. This paper also introduces the Ulimisana Optimisation Algorithm enabled Federated Learning (FL-UOA) framework which is an extension of the PBT-UOA. This framework is introduced to address the challenges of scattered datasets and privacy that is presented by the increase in connected end-devices. The FL-UOA learns on local data in scattered end-devices without sending datasets to a central server. The training datasets in local end-devices are used to evaluate models trained in other end-devices. The performance metrics are used to update the Social Trust Network (STN) of the FL-UOA framework. The FL-UOA outperformed the classic Federated Learning framework. This STN updating technique was tested in Machine Learning (ML) Unfairness to see how well it functioned as a regularisation term. This was achieved by training different models on subsets that contained datasets representing only specific sensitive groups. Results showed that by updating the hyperparameters while learning the parameters on the dataset scattered across different devices, the FL-UOA, takes advantage of diversified learning and reduces the ML Unfairness for models trained on group specific datasets. [ABSTRACT FROM AUTHOR]

Neuromorphic processors have gained momentum recently due to their high energy efficiency in artificial intelligence applications compared to DNN accelerators. Most neuromorphic processors are executing SNNs (Spiking Neural Networks). Liquid State Machine (LSM), as the spiking version of reservoir computing, shows advantages and great potential in image classification, speech recognition, language translation, etc.. Comparing with other SNN models, LSM has the characteristics of easy to train and low resource utilization, which is suitable for low-power and resource-constrained edge computing scenarios. In this paper, we propose a novel design of a neuromorphic processor, LSMCore, aiming at LSM acceleration. LSMCore supports both training and inference of LSM. It consists of 256 input neurons, 1024 liquid neurons, and 1.31M synapses. Besides, multiple optimization techniques, including weight quantization for reducing storage, zero-skipping for decreasing dynamic sparsity, and mini-batch training are adopted in this processor. The experimental results show that the frequency of LSMCore achieves 400 MHz, the power is 4.9W and the area is 18.49 mm2 with a 40nm library. Comparing with the baseline, LSMCore achieves up to $80.7\times $ ($49.6\times $), $91.3\times $ ($56.3\times $), and $83.1\times $ ($56.8\times $) speedup on MNIST, N-MNIST, and Free Spoken Digital Dataset (FSDD) respectively for training (inference), while the accuracy of LSMCore on these three datasets are 96.8%, 97.6%, and 90% respectively. [ABSTRACT FROM AUTHOR]

Purpose: This paper aims to propose a new assessment system module for handling the comprehensive answers written through the answer interface. Design/methodology/approach: The working principle is under three major phases: Preliminary semantic processing: In the pre-processing work, the keywords are extracted for each answer given by the course instructor. In fact, this answer is actually considered as the key to evaluating the answers written by the e-learners. Keyword and semantic processing of e-learners for hierarchical clustering-based ontology construction: For each answer given by each student, the keywords and the semantic information are extracted and clustered (hierarchical clustering) using a new improved rider optimization algorithm known as Rider with Randomized Overtaker Update (RR-OU). Ontology matching evaluation: Once the ontology structures are completed, a new alignment procedure is used to find out the similarity between two different documents. Moreover, the objects defined in this work focuses on "how exactly the matching process is done for evaluating the document." Finally, the e-learners are classified based on their grades. Findings: On observing the outcomes, the proposed model shows less relative mean squared error measure when weights were (0.5, 0, 0.5), and it was 71.78% and 16.92% better than the error values attained for (0, 0.5, 0.5) and (0.5, 0.5, 0). On examining the outcomes, the values of error attained for (1, 0, 0) were found to be lower than the values when weights were (0, 0, 1) and (0, 1, 0). Here, the mean absolute error (MAE) measure for weight (1, 0, 0) was 33.99% and 51.52% better than the MAE value for weights (0, 0, 1) and (0, 1, 0). On analyzing the overall error analysis, the mean absolute percentage error of the implemented RR-OU model was 3.74% and 56.53% better than k-means and collaborative filtering + Onto + sequential pattern mining models, respectively. Originality/value: This paper adopts the latest optimization algorithm called RR-OU for proposing a new assessment system module for handling the comprehensive answers written through the answer interface. To the best of the authors' knowledge, this is the first work that uses RR-OU-based optimization for developing a new ontology alignment-based online assessment of e-learners. [ABSTRACT FROM AUTHOR]

Conservation principles, such as conservation of charge, energy, or mass, provide a natural way to couple and constrain spatially separated variables. In this paper, we propose a dynamical system model that exploits these constraints for solving nonconvex and discrete global optimization problems. Unlike the traditional simulated annealing or quantum annealing-based global optimization techniques, the proposed method optimizes a target objective function by continuously evolving a driver functional over a conservation manifold, using a generalized variant of growth transformations. As a result, the driver functional asymptotically converges toward a Dirac-delta function that is centered at the global optimum of the target objective function. In this paper, we provide an outline of the proof of convergence for the dynamical system model and investigate different properties of the model using a benchmark nonlinear optimization problem. Also, we demonstrate how a discrete variant of the proposed dynamical system can be used for implementing decentralized optimization algorithms, where an ensemble of spatially separated entities (for example, biological cells or simple computational units) can collectively implement specific functions, such as winner-take-all and ranking, by exchanging signals only with its immediate substrate or environment. The proposed dynamical system model could potentially be used to implement continuous-time optimizers, annealers, and neural networks. [ABSTRACT FROM AUTHOR]

Abstract Environmental changes are traditionally considered intrinsic in evolutionary dynamic optimization. However, by ignoring that changes can instead be induced, we are ignoring that environmental changes can be eventually beneficial. To investigate the impact of artificial changes on the optimization speed up, we propose a framework for inducing artificial changes in any pseudo-Boolean or continuous optimization in this paper. Seven types of changes can be induced. Knowing when and how the changes occur allows us to design new strategies for evolutionary algorithms. Through computational experiments and illustrative examples, the impact of introducing changes in the optimization process is investigated. Experimental results indicate that changing the environments according to the proposed framework can lead to higher speed up, but not for all problems and change types. The best performance was obtained by change types that introduce plateaus and/or modify the gradient of regions of the fitness landscape around the current best solution. By doing this, the evolutionary dynamics is modified, eventually allowing the population to escape faster from local optima and reach new zones of the fitness landscape. Given a pseudo-Boolean or continuous optimization static problem, the proposed framework can be used to dynamically change the problem to speed up the optimization. [ABSTRACT FROM AUTHOR]

Purpose – The purpose of this paper is to solve the problem that the standard particle swarm optimization (PSO) algorithm has a low success rate when applied to the optimization of multi-dimensional and multi-extreme value functions, the authors would introduce the extended memory factor to the PSO algorithm. Furthermore, the paper aims to improve the convergence rate and precision of basic artificial fish swarm algorithm (FSA), a novel FSA optimized by PSO algorithm with extended memory (PSOEM-FSA) is proposed. Design/methodology/approach – In PSOEM-FSA, the extended memory for PSO is introduced to store each particle’ historical information comprising of recent places, personal best positions and global best positions, and a parameter called extended memory effective factor is employed to describe the importance of extended memory. Then, stability region of its deterministic version in a dynamic environment is analyzed by means of the classic discrete control theory. Furthermore, the extended memory factor is applied to five kinds of behavior pattern for FSA, including swarming, following, remembering, communicating and searching. Findings – The paper proposes a new intelligent algorithm. On the one hand, this algorithm makes the fish swimming have the characteristics of the speed of inertia; on the other hand, it expands behavior patterns for the fish to choose in the search process and achieves higher accuracy and convergence rate than PSO-FSA, owning to extended memory beneficial to direction and purpose during search. Simulation results verify that these improvements can reduce the blindness of fish search process, improve optimization performance of the algorithm. Research limitations/implications – Because of the chosen research approach, the research results may lack persuasion. In the future study, the authors will conduct more experiments to understand the behavior of PSOEM-FSA. In addition, there are mainly two aspects that the performance of this algorithm could be further improved. Practical implications – The proposed algorithm can be used to many practical engineering problems such as tracking problems. Social implications – The authors hope that the PSOEM-FSA can increase a branch of FSA algorithm, and enrich the content of the intelligent algorithms to some extent. Originality/value – The novel optimized FSA algorithm proposed in this paper improves the convergence speed and searching precision of the ordinary FSA to some degree. [ABSTRACT FROM AUTHOR]

Purpose – When facing a clouded global economy, many countries would increase their gold reserves. On the other hand, oil supply and demand depends on the political and economic situations of oil producing countries and their production technologies. Both oil and gold reserve play important roles in the economic development of a country. The paper aims to discuss this issue. Design/methodology/approach – This paper uses the historical data of oil and gold prices as research data, and uses the historical price tendency charts of oil and gold, as well as cluster analysis, to discuss the correlation between the historical data of oil and gold prices. By referring to the technical index equation of stocks, the technical indices of oil and gold prices are calculated as the independent variable and the closing price as the dependent variable of the forecasting model. Findings – The findings indicate that there is no obvious correlation between the price tendencies of oil and gold. According to five evaluating indicators, the MFOAGRNN forecast model has better forecast ability than the other three forecasting models. Originality/value – This paper explored the correlation between oil and gold prices, and built oil and gold prices forecasting models. In addition, this paper proposes a modified FOA (MFOA), where an escape parameter Δ is added to Si. The findings showed that the forecasting model that combines MFOA and GRNN has the best ability to forecast the closing price of oil and gold. [ABSTRACT FROM AUTHOR]

Purpose – The purpose of this paper is to propose a hybrid-simulated annealing algorithm to address the lacunas in production logistics. The primary focus is laid on the basic understanding of the critical quandary occurring in production logistics, and subsequently research attempts are undertaken to resolve the issue by developing a hybrid algorithm. A logistics problem associated with a flow shop (FS) having a string of jobs which need to be scheduled on m number of machines is considered. Design/methodology/approach – An attempt is made here to introduce and further establish a hybrid-simulated annealing algorithm (NEHSAO) with a new scheme for neighbourhood solutions generation, outside inverse (OINV). The competence in terms of performance of the proposed algorithm is enhanced by incorporating a fast polynomial algorithm, NEH, which provides the initial seed. Additionally, a new cooling scheme (Ex-Log) is employed to enhance the capacity of the algorithm. The algorithm is tested on the benchmark problems of Carlier and Reeves and subsequently validated against other algorithms reported in related literature. Findings – It is clearly observed that the performance of the proposed algorithm is far superior in most of the cases when compared to the other conventionally used algorithms. The proposed algorithm is then employed to a FS under dynamic conditions of machine breakdown, followed by formulation of three cases and finally identification of the best condition for scheduling under dynamic conditions. Originality/value – This paper proposes an hybrid algorithm to reduce makespan. Practical implementation of this algorithm in industries would lower the makespan and help the organisation to increse their profit [ABSTRACT FROM AUTHOR]

In this paper, a population-based algorithm inspired by the kidney process in the human body is proposed. In this algorithm the solutions are filtered in a rate that is calculated based on the mean of objective functions of all solutions in the current population of each iteration. The filtered solutions as the better solutions are moved to filtered blood and the rest are transferred to waste representing the worse solutions. This is a simulation of the glomerular filtration process in the kidney. The waste solutions are reconsidered in the iterations if after applying a defined movement operator they satisfy the filtration rate, otherwise it is expelled from the waste solutions, simulating the reabsorption and excretion functions of the kidney. In addition, a solution assigned as better solution is secreted if it is not better than the worst solutions simulating the secreting process of blood in the kidney. After placement of all the solutions in the population, the best of them is ranked, the waste and filtered blood are merged to become a new population and the filtration rate is updated. Filtration provides the required exploitation while generating a new solution and reabsorption gives the necessary exploration for the algorithm. The algorithm is assessed by applying it on eight well-known benchmark test functions and compares the results with other algorithms in the literature. The performance of the proposed algorithm is better on seven out of eight test functions when it is compared with the most recent researches in literature. The proposed kidney-inspired algorithm is able to find the global optimum with less function evaluations on six out of eight test functions. A statistical analysis further confirms the ability of this algorithm to produce good-quality results. [ABSTRACT FROM AUTHOR]

The diversity of solutions is of great importance for multi-objective evolutionary algorithms. In this paper, a new multi-objective particle swarm optimization algorithm based on decomposition (MPSO/D) is proposed. Firstly, the objective space of a multi-objective problem is decomposed into a set of sub-regions based on a set of direction vectors. Then MPSO/D makes each sub-region have a solution to maintain the diversity. Secondly, considering the convergence of solutions, MPSO/D uses the crowding distance to calculate the fitness values of the reserved solutions for selection operator, and uses the neighboring particles of a particle to determine the global best historical position ( gbest ) of the particle. The proposed algorithm has been compared with NSGAII, MOEA/D and NNIA on sixteen test sets. The experimental results illustrate that the proposed algorithm outperforms NSGAII, MOEA/D and NNIA in terms of convergence and diversity. [ABSTRACT FROM AUTHOR]

Sparse representation learning has been successfully applied into image classification, which represents a given image as a linear combination of an over-complete dictionary. The classification result depends on the reconstruction residuals. Normally, the images are stretched into vectors for convenience, and the representation residuals are characterized by l2 -norm or l1 -norm, which actually assumes that the elements in the residuals are independent and identically distributed variables. However, it is hard to satisfy the hypothesis when it comes to some structural errors, such as illuminations, occlusions, and so on. In this paper, we represent the image data in their intrinsic matrix form rather than concatenated vectors. The representation residual is considered as a matrix variate following the matrix elliptically contoured distribution, which is robust to dependent errors and has long tail regions to fit outliers. Then, we seek the maximum a posteriori probability estimation solution of the matrix-based optimization problem under sparse regularization. An alternating direction method of multipliers (ADMMs) is derived to solve the resulted optimization problem. The convergence of the ADMM is proven theoretically. Experimental results demonstrate that the proposed method is more effective than the state-of-the-art methods when dealing with the structural errors. [ABSTRACT FROM PUBLISHER]

Purpose -- Hyper-heuristics are a class of high-level search techniques which operate on a search space of heuristics rather than directly on a search space of solutions. The purpose of this paper is to investigate the suitability of using genetic programming as a hyper-heuristic methodology to generate constructive heuristics to solve the multidimensional 0-1 knapsack problem Design/methodology/approach -- Early hyper-heuristics focused on selecting and applying a low-level heuristic at each stage of a search. Recent trends in hyper-heuristic research have led to a number of approaches being developed to automatically generate new heuristics from a set of heuristic components. A population of heuristics to rank knapsack items are trained on a subset of test problems and then applied to unseen instances. Findings -- The results over a set of standard benchmarks show that genetic programming can be used to generate constructive heuristics which yield human-competitive results. Originality/value -- In this work the authors show that genetic programming is suitable as a method to generate reusable constructive heuristics for the multidimensional 0-1 knapsack problem. This is classified as a hyper-heuristic approach as it operates on a search space of heuristics rather than a search space of solutions. To our knowledge, this is the first time in the literature a GP hyper-heuristic has been used to solve the multidimensional 0-1 knapsack problem. The results suggest that using GP to evolve ranking mechanisms merits further future research effort. [ABSTRACT FROM AUTHOR]