Search

The paper develops Bayesian estimators and credible intervals for the reliability function and hazard rate function of Topp-Leone distribution under Type-II censored data by considering informative prior and non-informative prior using symmetric and asymmetric loss functions. Further, Monte Carlo simulation study is carried out to find the Bayes estimates for reliability function and hazard rate along with their credible intervals. Real life data set is also taken up for illustration purpose. Robustness of hyper-parameters is also discussed. [ABSTRACT FROM AUTHOR]

In December of 2019, a novel coronavirus (COVID-19) appeared in Wuhan city, China and has been reported in many countries with millions of people infected within only four months. Chest computed Tomography (CT) has proven to be a useful supplement to reverse transcription polymerase chain reaction (RT-PCR) and has been shown to have high sensitivity to diagnose this condition. Therefore, radiological examinations are becoming crucial in early examination of COVID-19 infection. Currently, CT findings have already been suggested as an important evidence for scientific examination of COVID-19 in Hubei, China. However, classification of patient from chest CT images is not an easy task. Therefore, in this paper, a deep bidirectional long short-term memory network with mixture density network (DBM) model is proposed. To tune the hyperparameters of the DBM model, a Memetic Adaptive Differential Evolution (MADE) algorithm is used. Extensive experiments are drawn by considering the benchmark chest-Computed Tomography (chest-CT) images datasets. Comparative analysis reveals that the proposed MADE-DBM model outperforms the competitive COVID-19 classification approaches in terms of various performance metrics. Therefore, the proposed MADE-DBM model can be used in real-time COVID-19 classification systems. [ABSTRACT FROM AUTHOR]

Since software system is becoming more and more complex than before, performance degradation and even abrupt download, which are called software aging phenomena, bring about a great deal of economic loss. To counter these problems, some methods are used. Support vector machine is an effective method to tackle software aging problems, but its performance is influenced by the selection of hyper-parameters. A method is proposed to optimize the hyper-parameter selection of support vector machine in this work. The proposed method which is used as a training algorithm to optimize the parameter selection of support vector machine, utilizes the global exploration power of firefly method to achieve faster convergence and also a better accuracy. In the experiment, we use two metrics to test the effect of the proposed method. The results indicate that the presented method owns the highest accuracy in both the available memory prediction and heap memory prediction of Web server for software aging predictions. [ABSTRACT FROM AUTHOR]

Gaussian process regression (GPR) is frequently used for uncertain measurement and prediction of nonstationary time series in the Internet of Things data, nevertheless, the generalization and regression efficacy of GPR are directly impacted by its selection of hyper-parameters. In the study, a non-inertial particle swarm optimization with elite mutation-Gaussian process regression (NIPSO-GPR) is proposed to optimize the hyper-parameters of GRP. NIPSO-GPR can adaptively obtain hyper-parameters of GPR via uniform non-inertial velocity update formula and adaptive elite mutation strategy. When compared with several frequently used algorithms of hyper-parameters optimization on linear and nonlinear time series sample data, experimental results indicate that GPR after hyper-parameters optimized by NIPSO-GPR has better fitting precision and generalization ability.

Accurate monitoring the surrounding environment is an important research direction in the field of unmanned systems such as bio-robotics, and has attracted much research attention in recent years. The trajectories of surrounding vehicles should be predicted accurately in space and time to realize active defense and running safety of an unmanned system. However, there is uncertainty and uncontrollability in the process of trajectory prediction of surrounding obstacles. In this study, we propose a trajectory prediction method based on a sequential model, that fuses two neural networks of a convolutional neural network (CNN) and a long short-term memory network (LSTM). First, a box plot is used to detect and eliminate abnormal values of vehicle trajectories, and valid trajectory data are obtained. Second, the trajectories of surrounding vehicles are predicted by merging the characteristics of CNN space expansion and LSTM time expansion; the hyper-parameters of the model are optimized according to a grid search algorithm, which satisfies the double-precision prediction requirement in space and time. Finally, data from next generation simulation (NGSIM) and Creteil roundabout in France are taken as test cases; the correctness and rationality of the method are verified by prediction error indicators. Experimental results demonstrate that the proposed CNN-LSTM method is more accurate and features a shorter time cost, which meets the prediction requirements and provides an effective method for the safe operation of unmanned systems. [ABSTRACT FROM AUTHOR]

Multispectral LiDAR (Light Detection And Ranging) is characterized of the completeness and consistency of its spectrum and spatial geometric data, which provides a new data source for land-cover classification. In recent years, the convolutional neural network (CNN), compared with traditional machine learning methods, has made a series of breakthroughs in image classification, object detection, and image semantic segmentation due to its stronger feature learning and feature expression abilities. However, traditional CNN models suffer from some issues, such as a large number of layers, leading to higher computational cost. To address this problem, we propose a CNN-based multi-spectral LiDAR land-cover classification framework and analyze its optimal parameters to improve classification accuracy. This framework starts with the preprocessing of multi-spectral 3D LiDAR data into 2D images. Next, a CNN model is constructed with seven fundamental functional layers, and its hyper-parameters are comprehensively discussed and optimized. The constructed CNN model with the optimized hyper-parameters was tested on the Titan multi-spectral LiDAR data, which include three wavelengths of 532 nm, 1064 nm, and 1550 nm. Extensive experiments demonstrated that the constructed CNN with the optimized hyper-parameters is feasible for multi-spectral LiDAR land-cover classification tasks. Compared with the classical CNN models (i.e., AlexNet, VGG16 and ResNet50) and our previous studies, our constructed CNN model with the optimized hyper-parameters is superior in computational performance and classification accuracies. [ABSTRACT FROM AUTHOR]

The selection of the hyper-parameters plays a critical role in the task of prediction based on the recurrent neural networks (RNN). Traditionally, the hyper-parameters of the machine learning models are selected by simulations as well as human experiences. In recent years, multiple algorithms based on Bayesian optimization (BO) are developed to determine the optimal values of the hyper-parameters. In most of these methods, gradients are required to be calculated. In this work, the particle swarm optimization (PSO) is used under the BO framework to develop a new method for hyper-parameter optimization. The proposed algorithm (BO-PSO) is free of gradient calculation and the particles can be optimized in parallel naturally. So the computational complexity can be effectively reduced which means better hyper-parameters can be obtained under the same amount of calculation. Experiments are done on real world power load data, where the proposed method outperforms the existing state-of-the-art algorithms, BO with limit-BFGS-bound (BO-L-BFGS-B) and BO with truncated-newton (BO-TNC), in terms of the prediction accuracy. The errors of the prediction result in different models show that BO-PSO is an effective hyper-parameter optimization method.

Highlights • A new fully automatic end-to-end deep learning model for brain tumor segmentation. • EnsembleNet uses Ensemble Learning to aggregate the results of two models which are based on an incremental deep neural network. • We present a new training strategy which unifies the training methods of deep learning models. • The new method of designing deep learning models with the advantage of GPU implementation, our models are one order of magnitude faster compared to the state-of-the-art. Graphical abstract Abstract Background and Objective : Nowadays, getting an efficient Brain Tumor Segmentation in Multi-Sequence MR images as soon as possible, gives an early clinical diagnosis, treatment and follow-up. The aim of this study is to develop a new deep learning model for the segmentation of brain tumors. The proposed models are used to segment the brain tumors of Glioblastomas (with both high and low grade). Glioblastomas have four properties: different sizes, shapes, contrasts, in addition, Glioblastomas appear anywhere in the brain. Methods : In this paper, we propose three end-to-end Incremental Deep Convolutional Neural Networks models for fully automatic Brain Tumor Segmentation. Our proposed models are different from the other CNNs-based models that follow the technique of trial and error process which does not use any guided approach to get the suitable hyper-parameters. Moreover, we adopt the technique of Ensemble Learning to design a more efficient model. For solving the problem of training CNNs model, we propose a new training strategy which takes into account the most influencing hyper-parameters by bounding and setting a roof to these hyper-parameters to accelerate the training. Results : Our experiment results reported on BRATS-2017 dataset. The proposed deep learning models achieve the state-of-the-art performance without any post-processing operations. Indeed, our models achieve in average 0.88 Dice score over the complete region. Moreover, the efficient design with the advantage of GPU implementation, allows our three deep learning models to achieve brain segmentation results in average 20.87 s. Conclusions : The proposed deep learning models are effective for the segmentation of brain tumors and allow to obtain high accurate results. Moreover, the proposed models could help the physician experts to reduce the time of diagnostic. [ABSTRACT FROM AUTHOR]

This study was conducted on the presentation of a method to improve the forecast of urban gas consumption based on the weather variables including temperature, pressure, humidity, wind speed and also the gas price. The diversity of input variables as well as investigating a short-term (daily) scale, led to creation complex and nonlinear relationships between the variables, which makes its solving difficult. To this end, the categorical boosting (CatBoost) method is combined with some meta-heuristic algorithms to create hybrid models. These meta-heuristic algorithms include Phasor Particle Swarm Optimization, Artificial Bee Colony, Battle Royale Optimizer, Grey Wolf Optimizer, Satin Bowerbird algorithm, and Fruit Fly Optimization Algorithm. During the network training, the K-Fold cross-validation has also been used to prevent overfitting. Finally, using an actual dataset, the performance of the proposed method is investigated. The results showed that the proposed method can predict the value of short-term urban gas consumption. The results showed that the hybrid Catboost-PPSO model had the best performance among all presented hybrid models. Therefore, using the PPSO algorithm to optimize the hyper-parameters of the CatBoost network is recommended for predicting gas consumption. • The study was performed on the forecast of urban gas consumption. • Meta-heuristic algorithms help CatBoost network to perform more efficiently. • The results showed that the hybrid Catboost-PPSO model has the best performance. • The results showed that the hybrid Catboost-FOA model performs weaker than others. • In training and testing, the TIC values of the hybrid Catboost-PPSO model are 0.0013 and 0.0540. [ABSTRACT FROM AUTHOR]

The thermal comfort indices are usually identified using empirical thermal models based on the human balanced equations and experimentations. In our paper, we propose a statistical regression method to predict these indices. To achieve this goal, first, the fuzzy support vector regression (FSVR) identification approach was integrated with the particle swarm optimization (PSO) algorithm. Then PSO was used as a global optimizer to optimize and select the hyper-parameters needed for the FSVR model. The radial basis function (RBF) kernel was used within the FSVR model. Afterward, these optimal hyper-parameters were used to forecast the thermal comfort indices: predicted mean vote (PMV), predicted percentage dissatisfied (PPD), new standard effective temperature (SET*), thermal discomfort (DISC), thermal sensation (TSENS) and predicted percent dissatisfied due to draft (PD). The application of the proposed approach on different data sets gave successful prediction and promising results. Moreover, the comparisons between the traditional Fanger model and the new model further demonstrate that the proposed model achieves even better identification performance than the original FSVR technique. [ABSTRACT FROM AUTHOR]

The photovoltaic grid connection can impact the power grid and affect its stability; therefore, making predictions about photovoltaic power is critically important for the grid scheduling department to properly plan power generation. The characteristics of photovoltaic power are analyzed, and the principle of sparse Bayesian regression is studied; thus, a photovoltaic power prediction model based on the sparse Bayesian regression algorithm is established. Traditional sparse Bayesian regression uses the maximum likelihood method to optimize hyper-parameters, which has some disadvantages, for example, the optimization effect excessively depends on initial values and iterations are difficult to determine. In this article, the artificial bee colony is used instead of the maximum likelihood method to optimize the hyper-parameters. An improved sparse Bayesian regression model based on artificial bee colony optimization is proposed that considers meteorological factors and historical power data. Finally, the state grid Scenery Storage Demonstration Project data are used to test the proposed prediction model. The simulation result shows that the improved sparse Bayesian regression model achieves good prediction effects. [ABSTRACT FROM AUTHOR]

Latent Dirichlet allocation (LDA) has seen increasing use in the understanding of source code and its related artifacts in part because of its impressive modeling power. However, this expressive power comes at a cost: The technique includes several tuning parameters whose impact on the resulting LDA model must be carefully considered. The aim of this work is to provide insights into the tuning parameters' impact. Doing so improves the comprehension of both researchers who look to exploit the power of LDA in their research and those who interpret the output of LDA-using tools. It is important to recognize that the goal of this work is not to establish values for the tuning parameters because there is no universal best setting. Rather, appropriate settings depend on the problem being solved, the input corpus (in this case, typically words from the source code and its supporting artifacts), and the needs of the engineer performing the analysis. This work's primary goal is to aid software engineers in their understanding of the LDA tuning parameters by demonstrating numerically and graphically the relationship between the tuning parameters and the LDA output. A secondary goal is to enable more informed setting of the parameters. Copyright © 2016 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

Rapidly and accurately modeling of microbial fuel cells (MFCs) plays an important role not only in thorough understanding of the effects of operating conditions on system performance, but also in the successful implementation of real-time maximization of power output. Although the first principle electrochemical model has better generalization performance, it is often time-consuming for model construction and is hard to real-time application. In this study, a nonparametric Gaussian process regression (GPR) model is used to capture the nonlinear relationship between operating conditions and output voltage in the MFCs. A simple online learning strategy is proposed to recursively update the hyper-parameters of the GPR model. The applicability and effectiveness of the proposed method is validated by both the simulation and experimental datasets from the acetate and the glucose and glutamic acid two-chamber MFCs. The results illustrate that the online GPR model provides a promising method for capturing the complex nonlinearity phenomenon in MFCs, which can be greatly helpful for further real-time optimization of MFCs. [ABSTRACT FROM AUTHOR]

• Building IDS in non-stationary environment. • Proposed a module to track the changing optima for hyper-parameters. • Proposed knowledge transfer mechanism that can be used with metaheuristic optimization algorithms. • Minimize the average optimization time. • Proposed framework helps is retaining the performance of IDS in non-stationary environment. Building an Intrusion Detection System (IDS) in non-stationary environment is challenging because, in such an environment, intrusion-related data grow every day. A machine learning model trained in a stationary environment where training data does not change, often fails to retain its performance in real world environment. This is because dynamism in data makes the hyper-parametric values of the underlying classifier shift in the search space. For making such a model work for intrusion detection in non-stationary environment, one must have to run hyper-parametric optimization algorithm again and again at various time instances. But the expansion of the existing data in non-stationary environment, makes such a way of tunning the hyper-parameters computationally expensive. So, there is a requirement of more adaptive and computationally efficient optimization frameworks for hyper-parameters to build IDS in non-stationary environment. This paperwork proposes a novel framework to train a Support Vector Machine (SVM) for intrusion detection by optimizing its hyper-parameters C and γ dynamically. For designing this framework, we have used Moth-Flame Optimization (MFO) as the base optimization algorithm which can be run with random initialization. Further, for utilizing the knowledge generated by running the base optimization algorithm, we have introduced two algorithms- a Lightweight MFO and a simple Knowledge-Based Search. The Lightweight MFO uses the knowledge for initializing the starting solutions and the Knowledge-Based Search uses the knowledge as search space. Based on the result of a drift detection module, the proposed framework identifies the most appropriate algorithm to be used at a particular time instance when re-training of the model is required due to the change in the data. Results have shown a significant reduction in the average time complexity of the hyper-parametric optimization process. We have evaluated our proposed framework on benchmark NSL-KDD dataset and got significantly encouraging convergence rate and detection performance. The obtained average accuracy for IDS built using our proposed framework is 97.5%. Further , we have also compared our framework by considering other metaheuristic algorithms as base optimization algorithms and found that our proposed framework, which uses MFO as a base optimization algorithm outperforms the others. [ABSTRACT FROM AUTHOR]

A nonlinear multivariable model of a locomotive proton exchange membrane fuel cell (PEMFC) system based on a support vector regression (SVR) is proposed to study the effect of different operating conditions on dynamic behavior of a locomotive PEMFC power unit. Furthermore, an effective informed adaptive particle swarm optimization (EIA-PSO) algorithm which is an adaptive swarm intelligence optimization with preferable search ability and search rate is utilized to tune the hyper-parameters of the SVR model for the improvement of model performance. The comparisons with the experimental data demonstrate that the SVR model based on EIA-PSO can efficiently approximate the dynamic behaviors of locomotive PEMFC power unit and is capable of predicting dynamic performance in terms of the output voltage and power with a high accuracy. [ABSTRACT FROM AUTHOR]

This paper studies the problem of hyper-parameters selection for a linear programming-based support vector machine for regression (LP-SVR). The proposed model is a generalized method that minimizes a linear-least squares problem using a globalization strategy, inexact computation of first order information, and an existing analytical method for estimating the initial point in the hyper-parameters space. The minimization problem consists of finding the set of hyper-parameters that minimizes any generalization error function for different problems. Particularly, this research explores the case of two-class, multi-class, and regression problems. Simulation results among standard data sets suggest that the algorithm achieves statistically insignificant variability when measuring the residual error; and when compared to other methods for hyper-parameters search, the proposed method produces the lowest root mean squared error in most cases. Experimental analysis suggests that the proposed approach is better suited for large-scale applications for the particular case of an LP-SVR. Moreover, due to its mathematical formulation, the proposed method can be extended in order to estimate any number of hyper-parameters. [ABSTRACT FROM AUTHOR]

This paper addresses the Bayesian calibration of dynamic models with parametric and structural uncertainties, in particular where the uncertain parameters are unknown/poorly known spatio-temporally varying subsystem models. Independent stationary Gaussian processes with uncertain hyper-parameters describe uncertainties of the model structure and parameters, while Karhunen-Loeve expansion is adopted to spectrally represent these Gaussian processes. The Karhunen-Loeve expansion of a prior Gaussian process is projected on a generalized Polynomial Chaos basis, whereas intrusive Galerkin projection is utilized to calculate the associated coefficients of the simulator output. Bayesian inference is used to update the prior probability distribution of the generalized Polynomial Chaos basis, which along with the chaos expansion coefficients represent the posterior probability distribution. The proposed method is demonstrated for calibration of a simulator of quasi-one-dimensional flow through a divergent nozzle with uncertain nozzle area profile. The posterior distribution of the nozzle area profile and the hyper-parameters obtained using the proposed method are found to match closely with the direct Markov Chain Monte Carlo-based implementation of the Bayesian framework. Efficacy of the proposed method is demonstrated for various choices of prior. Posterior hyper-parameters of the model structural uncertainty are shown to quantify acceptability of the simulator model. [ABSTRACT FROM AUTHOR]

Abstract: Spatio-temporal estimation of precipitation over a region is essential to the modeling of hydrologic processes for water resources management. The changes of magnitude and space–time heterogeneity of rainfall observations make space–time estimation of precipitation a challenging task. In this paper we propose a Box–Cox transformed hierarchical Bayesian multivariate spatio-temporal interpolation method for the skewed response variable. The proposed method is applied to estimate space–time monthly precipitation in the monsoon periods during 1974–2000, and 27-year monthly average precipitation data are obtained from 51 stations in Pakistan. The results of transformed hierarchical Bayesian multivariate spatio-temporal interpolation are compared to those of non-transformed hierarchical Bayesian interpolation by using cross-validation. The software developed by [11] is used for Bayesian non-stationary multivariate space–time interpolation. It is observed that the transformed hierarchical Bayesian method provides more accuracy than the non-transformed hierarchical Bayesian method. [Copyright &y& Elsevier]

Metabolites can serve as biomarkers. Estimation of metabolite concentrations from an in vivo magnetic resonance spectroscopy (MRS) signal often uses a reference signal to estimate a model function of the spectral lineshape. When no reference signal is available, the a priori unknown in vivo lineshape must be inferred from the data at hand. This makes quantitation of metabolites from in vivo MRS signals a semi-parametric estimation problem which, in turn, implies setting of hyper-parameters by users of the software involved. Estimation of metabolite concentrations is usually done by nonlinear least-squares (NLLS) fitting of a physical model function based on minimizing the residue. In this work, the semi-parametric task is handled by complementing the usual criterion of minimal residue with a second criterion acting in tandem with it. This second criterion is derived from the general physical knowledge that the width of the line is limited. The limit on the width is a hyper-parameter; its setting appeared not critical so far. The only other hyper-parameter is the relative weight of the two criteria. But its setting too is not critical. Attendant estimation errors, obtained from a Monte Carlo calculation, show that the two-criterion NLLS approach successfully handles the semi-parametric aspect of metabolite quantitation. [ABSTRACT FROM AUTHOR]

We propose simple parametric and nonparametric bootstrap methods for estimating the prediction mean square error (PMSE) of state vector predictors that use estimated model parameters. As is well known, substituting the model parameters by their estimates in the theoretical PMSE expression that assumes known parameter values results in underestimation of the true PMSE. The parametric method consists of generating parametrically a large number of bootstrap series from the model fitted to the original series, re-estimating the model parameters for each series using the same method as used for the original series and then estimating the separate components of the PMSE. The nonparametric method generates the series by bootstrapping the standardized innovations estimated for the original series. The bootstrap methods are compared with other methods considered in the literature in a simulation study that also examines the robustness of the various methods to non-normality of the model error terms. Application of the bootstrap method to a model fitted to employment ratios in the USA that contains 18 unknown parameters, estimated by a three-step procedure yields unbiased PMSE estimators. [ABSTRACT FROM AUTHOR]

Priors are seldom unequivocal and an important component of Bayesian modeling is assessment of the sensitivity of the posterior to the specified prior distribution. This is especially true in fisheries science where the Bayesian approach has been promoted as a rigorous method for including existing information from previous surveys and from related stocks or species. These informative priors may be highly contested by various interest groups. Here, formulae for the first and second derivatives of Bayes estimators with respect to hyper-parameters of the joint prior density are given. The formula for the second derivative provides a correction to a previously published result. The formulae are shown to reduce to very convenient and easily implemented forms when the hyper-parameters are for exponential family marginal priors. For model parameters with such priors it is shown that the ratio of posterior variance to prior variance can be interpreted as the sensitivity of the posterior mean to the prior mean. This methodology is applied to a nonlinear state-space model for the biomass of South Atlantic albacore tuna and sensitivity of the maximum sustainable yield to the prior specification is examined. [ABSTRACT FROM AUTHOR]

A Bayesian approach is developed to assess the factor analysis model. Joint Bayesian estimates of the factor scores and the structural parameters in the covariance structure are obtained simultaneously. The basic idea is to treat the latent factor scores as missing data and augment them with the observed data in generating a sequence of random observations from the posterior distributions by the Gibbs sampler. Then, the Bayesian estimates are taken as the sample means of these random observations. Expressions for implementing the algorithm are derived and some statistical properties of the estimates are presented. Some aspects of the algorithm are illustrated by a real example and the performance of the Bayesian procedure is studied using simulation. [ABSTRACT FROM AUTHOR]

• We proposed a novel model hyOPTXg to identify cardiovascular disease. • The proposed model hyOPTXg utilized the XGBoost and OPTUNA techniques. • Data pre-processing and hyper-parameter tuning techniques were applied. Cardiovascular disease is a dangerous disorder that causes the most significant number of deaths across the world. In the past years, researchers proposed several automated systems to identify heart disease early so that there is a chance to enhance the diagnosis process. This research paper proposes an expert model called hyOPTXg, which predicts heart disease using an optimized XGBoost classifier. To produce a better system using a classifier, we need decent hyper-parameter tuning. So, we tuned the hyper-parameters of XGBoost and trained the model using tuned parameters. The framework used for hyper-parameter tuning is OPTUNA (hyper-parameter optimization technique). This system was tested on three datasets: The Cleveland dataset and the Heart Failure prediction dataset from Kaggle and heart disease UCI from Kaggle. We used various metrics to assess the system's efficiency, including recall, precision, f1-score, accuracy, and the ROC chart's area under the curve (AUC). And we got better results compared to other systems which other authors propose in recent times. We got 94.7% in the Cleveland dataset,89.3% in the Kaggle heart failure dataset, and 88.5% in the heart disease UCI Kaggle dataset. Our prediction result says that we got better results than other systems proposed by the other authors. And we did a comparison study of our model with other authors' models. [ABSTRACT FROM AUTHOR]

• The NASA dataset is trained with a tuned 2-layer LSTM and GRU models. • The results are compared to existing outputs and inference shown. • Introduce a semi-novel optimization algorithm using genetic algorithms. • Hyper-parameters of the model – learning rate and batch size are self-tuned by a genetic algorithm. The application of remaining useful life (RUL) prediction is very important in terms of energy optimization, cost-effectiveness, and risk mitigation. The existing RUL prediction algorithms mostly constitute deep learning frameworks. In this paper, we implement LSTM and GRU models and compare the obtained results with a proposed genetically trained neural network. The current models solely depend on ADAM and SGD for optimization and learning. Although the models have worked well with these optimizers, even little uncertainties in prognostics prediction can result in huge losses. We hope to improve the consistency of the predictions by adding another layer of optimization using Genetic Algorithms. The hyper-parameters – learning rate and batch size are optimized beyond manual capacity. These models and the proposed architecture are tested on the NASA Turbofan Jet Engine dataset. The optimized architecture can predict the given hyper-parameters autonomously and provide superior results. [ABSTRACT FROM AUTHOR]

The selection of the hyper-parameters plays a critical role in the task of prediction based on the recurrent neural networks (RNN). Traditionally, the hyper-parameters of the machine learning models are selected by simulations as well as human experiences. In recent years, multiple algorithms based on Bayesian optimization (BO) are developed to determine the optimal values of the hyper-parameters. In most of these methods, gradients are required to be calculated. In this work, the particle swarm optimization (PSO) is used under the BO framework to develop a new method for hyper-parameter optimization. The proposed algorithm (BO-PSO) is free of gradient calculation and the particles can be optimized in parallel naturally. So the computational complexity can be effectively reduced which means better hyper-parameters can be obtained under the same amount of calculation. Experiments are done on real world power load data, where the proposed method outperforms the existing state-of-the-art algorithms, BO with limit-BFGS-bound (BO-L-BFGS-B) and BO with truncated-newton (BO-TNC), in terms of the prediction accuracy. The errors of the prediction result in different models show that BO-PSO is an effective hyper-parameter optimization method. [ABSTRACT FROM AUTHOR]

• Considered dynamic nature of data for real world problems. • Proposed shift detection for shifting optimum. • Proposed knowledge based search with MFO. • Fast convergence rate to minimize execution time. • Proposed framework is highly inclined towards accuracy of SVM. In the real world, most of the collections of data are dynamic in nature, i.e. their size may grow with time. This dynamic nature of the data not only reduces the performance of the classifiers but also demands more optimized models for retaining the performance. Due to this, machine learning models developed in a static environment cannot be deployed efficiently to solve the real-world problems. Nowadays, maximum existing works consider only the static behaviour of the data for the training of machine learning models where the size of the collection of training data does not change over time. This paperwork imposes Support Vector Machine (SVM) in a dynamic environment. It has been identified that shifting of the optimum values of two hyper-parameters C (Penalty Parameter) and γ (Kernel Parameter) in the search space is one of the primary reasons for the performance degradation of SVM in dynamic environment. This paper proposes a novel framework that uses a new optimization module Knowledge-Based-Search (KBS) along with Moth –Flame Optimization (MFO) to optimize C and γ in a dynamic environment to train SVM efficiently. KBS uses knowledge gathered at various instances of time, which are the bi-products of MFO. MFO in our framework is the base optimization algorithm which works underneath KBS. The experiments have shown that KBS helps in controlling the exponential growth of the time complexity of the optimization process where only MFO is used to optimize C and γ. Integration of KBS with MFO brings down the time complexity to a large extent. To validate the proposed framework we have used a simulated dynamic environment for profit/loss classification problem for organizations. The experiments have also shown that KBS's integration with MFO outperforms integration of KBS with other modern optimization techniques such as Particle Swarm Optimization (PSO), Multi-Verse Optimization (MVO), Grey-Wolf Optimization (GWO), Cuckoo Search (CS), Whale Optimization Algorithm (WOA), Genetic Algorithm (GA), Fire-Fly Algorithm (FFA) and Salp Swarm Algorithm (SSA). [ABSTRACT FROM AUTHOR]

Deep Belief Networks (DBN) have become a powerful tools to deal with a wide range of applications. On complex tasks like image reconstruction, DBN's performance is highly sensitive to parameter settings. Manually trying out different parameters is tedious and time consuming however often required in practice as there are not many better options. This work proposes an evolutionary hyper-heuristic framework for automatic parameter optimisation of DBN. The hyper-heuristic framework introduced here is the first of its kind in this domain. It involves a high level strategy and a pool of evolutionary operators such as crossover and mutation to generates DBN parameter settings by perturbing or modifying the current setting of a DBN. Providing a large set of operators could be beneficial to form a more effective high level strategy, but in the same time would increase the search space hence make it more difficulty to form a good strategy. To address this issue, a non-parametric statistical test is introduced to identify a subset of effective operators for different phases of the hyper-heuristic search. Three well-known image reconstruction datasets were used to evaluate the performance of the proposed framework. The results reveal that the proposed hyper-heuristic framework is very competitive when compared to the state of art methods. • We propose a hyper-heuristic to optimise Deep Belief Networks. • We use a large set of heuristic with in the hyper-heuristic. • We propose a non-parametric statistical test to identify the best heuristics. • We tested the proposed hyper-heuristics on three different datasets. • The results demonstrated the effectiveness of the hyper-heuristic. [ABSTRACT FROM AUTHOR]