Search

WOS:000903228300002 PubMed ID:36573103 Extreme learning machine (ELM) as a type of feedforward neural network has been widely used to obtain beneficial insights from various disciplines and real-world applications. Despite the advantages like speed and highly adaptability, instability drawbacks arise in case of multicollinearity, and to overcome this, additional improvements were needed. Regularization is one of the best choices to overcome these drawbacks. Although ridge and Liu regressions have been considered and seemed effective regularization methods on ELM algorithm, each one has own characteristic features such as the form of tuning parameter, the level of shrinkage or the norm of coefficients. Instead of focusing on one of these regularization methods, we propose a combination of ridge and Liu regressions in a unified form for the context of ELM as a remedy to aforementioned drawbacks. To investigate the performance of the proposed algorithm, comprehensive comparisons have been carried out by using various real-world data sets. Based on the results, it is obtained that the proposed algorithm is more effective than the ELM and its variants based on ridge and Liu regressions, RR-ELM and Liu-ELM, in terms of the capability of generalization. Generalization performance of proposed algorithm on ELM is remarkable when compared to RR-ELM and Liu-ELM, and the generalization performance of the proposed algorithm on ELM increases as the number of nodes increases. The proposed algorithm outperforms ELM in all data sets and all node numbers in that it has a smaller norm and standard deviation of the norm. Additionally, it should be noted that the proposed algorithm can be applied for both regression and classification problems.

In this paper, we introduce stochastic-restricted Liu predictors which will be defined by combining in a special way the two approaches followed in obtaining the mixed predictors and the Liu predictors in the linear mixed models. Superiorities of the linear combination of the new predictor to the Liu and mixed predictors are done in the sense of mean square error matrix criterion. Finally, numerical examples and a simulation study are done to illustrate the findings. In numerical examples, we took some arbitrary observations from the data as the prior information since we did not have historical data or additional information about the data sets. The results show that this case does the new estimator gain efficiency over the constituent estimators and provide accurate estimation and prediction of the data.

The purpose of this article is to obtain the jackknifed ridge predictors in the linear mixed models and to examine the superiorities, the linear combinations of the jackknifed ridge predictors over the ridge, principal components regression, r-k class and Henderson's predictors in terms of bias, covariance matrix and mean square error criteria. Numerical analyses are considered to illustrate the findings and a simulation study is conducted to see the performance of the jackknifed ridge predictors. © 2019, © 2019 Informa UK Limited, trading as Taylor & Francis Group.

In this study, we settle on the issue that when multicollinearity and unusual observations arise simultaneously and we straightforwardly extend leverages, Pearson residuals, delta beta and delta chi-square statistics using the principal components logistic regression (PCLR) estimator where the extensions typically take the advantage of the computation of PCLR estimator by one-step approximation. We then applied two simulation studies and a numerical example to illustrate the behavior of statistics for the PCLR estimator versus the traditional ML estimator. © 2019, © 2019 Taylor & Francis Group, LLC.

In order to combat multicollinearity, the r–d class estimator was introduced in linear and binary logistic regression models. Since the generalized linear models (GLMs) are the models that include logistic regression model, Poisson regression model, etc., we introduce the iterative and first-order approximated r–d class estimator in GLMs. The sampling distribution of the r–d class estimator at convergence is given and the test on the regression coefficients is provided. The properties of the first-order approximated r–d class estimator in GLMs are discussed, comparisons of the three constitute estimators in the sense of bias, variance and scalar mean square errors are done, and a cross-validation method and a mean square error method for the selection of the shrinkage parameter are given. Finally, a simulation study is conducted for Poisson response and two real data analyses are done for gamma and binomial response data to examine the performance of the first-order approximated r–d class estimator versus the first-order approximated maximum likelihood, Liu, PCR and ridge estimators in GLMs. © 2019, © 2019 Informa UK Limited, trading as Taylor & Francis Group.

In this paper, we focus on the progress of variant of conceptual predictive (Cp) statistic and we propose the model selection criterion that depend on Cp statistic under ridge regression for linear mixed model selection. The proposed criterion is conditional ridge Cp (CRCp) statistic based on the expected conditional Gauss discrepancy. Two versions of CRCp statistic under the assumptions that the variance components are known and unknown are derived. To examine the performance of the proposed criterion, a real data analysis and a Monte Carlo simulation study are given. © 2018, © 2018 Informa UK Limited, trading as Taylor & Francis Group.

This paper is concerned with the ridge estimation of fixed and random effects in the context of Henderson's mixed model equations in the linear mixed model. For this purpose, a penalized likelihood method is proposed. A linear combination of ridge estimator for fixed and random effects is compared to a linear combination of best linear unbiased estimator for fixed and random effects under the mean-square error (MSE) matrix criterion. Additionally, for choosing the biasing parameter, a method of MSE under the ridge estimator is given. A real data analysis is provided to illustrate the theoretical results and a simulation study is conducted to characterize the performance of ridge and best linear unbiased estimators approach in the linear mixed model. © 2016 Informa UK Limited, trading as Taylor & Francis Group.

This article is concerned with the predictions in linear mixed models under stochastic linear restrictions. Mixed and stochastic restricted ridge predictors are introduced by using Gilmour's approach. We also investigate assumptions that the variance parameters are not known under stochastic linear restrictions and attain estimators of variance parameters. Superiorities the linear combinations of the predictors are done in the sense of mean square error matrix criterion. Finally, a hypothetical data set is considered to illustrate the findings. © 2016 Elsevier Inc. FDK-2015-3968 This research was supported by Research Fund of Çukurova University under Project Number FDK-2015-3968 . Appendix A

Multicollinearity among the explanatory variables seriously effects the maximum likelihood estimator in linear regression models which results in too large estimates in absolute value and in large variance-covariance matrix. The adverse effects of multicollinearity on parameter estimation in generalized linear models are also explored by various authors in the case of maximum likelihood estimation. In this study, we introduce a first-order approximated Liu estimator to combat multicollinearity in generalized linear models which is an extension of the Liu estimator in the linear regression model. We also obtain necessary and sufficient condition for the superiority of the first-order approximated Liu estimator over the the first-order approximated maximum likelihood estimator by the approximated mean squared error criterion. Biasing parameter estimator of the first-order approximated Liu estimator is proposed by minimizing the approximated mean squared error. The results are illustrated by conducting a numerical example and simulation study. © 2016, Springer-Verlag Berlin Heidelberg.

Logistic regression is a widely used method to model categorical response data, and maximum likelihood (ML) estimation has widespread use in logistic regression. Although ML method is the most used method to estimate the regression coefficients in logistic regression model, multicollinearity seriously affects the ML estimator. To remedy the undesirable effects of multicollinearity, estimators alternative to ML are proposed. Drawing on the similarities between the multiple linear and logistic regressions, ridge, Liu and two parameter estimators are proposed which are based on the ML estimator. On the other hand, first-order approximated ridge estimator is proposed for use in logistic regression. This study will present further solutions to the problem in the form of alternative estimators which reduce the effect of collinearity. Owing to this, first-order approximated Liu, iterative Liu and iterative two parameter estimators are proposed. A simulation study as well as real life application are carried out to ascertain the effect of sample size and degree of multicollinearity, in which the ML based, first-order approximated and iterative biased estimators are compared. Graphical representations are presented which support the effect of the shrinkage parameter on the mean square error and prediction mean square error of the biased estimators. © 2016, Springer-Verlag Berlin Heidelberg.

Many researchers have studied restricted estimation in the context of exact and stochastic restrictions in linear regression. Some ideas in linear regression, where the ridge and restricted estimations are the well known, were carried to the generalized linear models which provide a wide range of models, including logistic regression, Poisson regression, etc. This study considers the estimation of generalized linear models under stochastic restrictions on the parameters. Furthermore, the sampling distribution of the estimators under the stochastic restriction, the compatibility test and choice of the biasing parameter are given. A real data set is analyzed and simulation studies concerning Binomial and Poisson distributions are conducted. The results show that when stochastic restrictions and ridge idea are simultaneously applied to the estimation methods, the new estimator gains efficiency in terms of having smaller variance and mean square error. © 2019, Springer-Verlag GmbH Germany, part of Springer Nature. FBI-2017-9024 This work was supported by Research Fund of Çukurova University under Project Number FBI-2017-9024 to M. Revan Özkale, a visiting scholar for one month at Stockholm University, Department of Statistics.

The purpose of this paper is to solve the problem of multicollinearity that affects the estimation of logistic regression model by introducing first-order approximated jackknifed ridge logistic estimator which is more efficient than the first-order approximated maximum likelihood estimator and has smaller variance than the first-order approximated jackknife ridge logistic estimator. Comparisons of the first-order approximated jackknifed ridge logistic estimator to the first-order approximated maximum likelihood, first-order approximated ridge, first-order approximated r-k class and principal components logistic regression estimators according to the bias, covariance and mean square error criteria are done. Three different estimators for the ridge parameter are also proposed. A real data set is used to see the performance of the first-order approximated jackknifed ridge logistic estimator over the first-order approximated maximum likelihood, first-order approximated ridge logistic, first-order approximated r-k class and first-order approximated principal components logistic regression estimators. Finally, two simulation studies are conducted in order to show the performance of the first-order approximated jackknife ridge logistic estimator. © 2018, Springer-Verlag GmbH Germany, part of Springer Nature.

Investigators that seek to employ regression analysis usually encounter the problem of multicollinearity with dependency on two or more explanatory variables. Multicollinearity is associated with unstable estimated coefficients and it results in high variances of the least squares estimators in linear regression models (LRMs). Thus the detection of collinearity is the compulsory first step in regression analysis. Multicollinearity also come out in generalized linear models (GLMs) and has same serious effects on the maximum likelihood estimates. The purposes of this paper are to propose new collinearity diagnostics criteria in GLMs in the context of both the maximum likelihood and ridge estimators, to examine the properties of new collinearity diagnostics via the ridge constant, to exemplify the theoretical results by numerical examples on Poisson, Binomial and Gamma responses. The effects of centering and scaling the information matrix on the sensitivity of the diagnostics in the presence of collinearity are also investigated. © 2019, © 2019 Taylor & Francis Group, LLC.

In this study, we introduce iterative restricted Liu estimator to combat multicollinearity in generalized linear models. We also obtain necessary and sufficient conditions for the superiority of the first-order approximated restricted Liu estimator over the first-order approximated maximum likelihood and Liu estimators by the approximated mean squared error criterion. The results are illustrated by conducting simulation studies and numerical examples. © 2019 Hacettepe University. All rights reserved.

In this paper, the (Formula presented.) class logistic estimator which combines the ridge logistic and principal components logistic regression (PCLR) estimators is proposed. The comparisons of the (Formula presented.) class logistic, ridge logistic and PCLR estimators to the maximum likelihood (ML) estimator according to the asymptotic matrix mean square error (MSE) criterion and the comparisons of the (Formula presented.) class logistic estimator to the ML, PCLR and ridge logistic estimators in the sense of asymptotic scalar MSE criterion are done. The results are illustrated with numerical examples and a simulation study is done. © 2016 Taylor & Francis.

The adverse effects of multicollinearity and unusual observations are seen in logistic regression and attention had been given in the literature to each of these problems separately. However, multicollinearity and unusual observations can arise simultaneously in logistic regression. The objective of this paper is to propose the statistics for detecting the unusual observations in an ill-conditioned data set under the ridge logistic estimator. A numerical example and two Monte Carlo simulation studies are used to illustrate the methodology. The present investigation shows that ridge logistic estimation copes with unusual observations by downweighting their influence. © 2017, Springer-Verlag GmbH Germany.

In this article, we propose the principal components regression and r-k class predictors, which combine the techniques of the ridge and principal components regressions in the linear mixed models. We demonstrate that the Henderson's predictors, the ridge predictors and the principal components regression predictors are special cases of the r-k class predictors. We also research assumption that the variance parameters are not known and get estimators of variance parameters. The necessary and sufficient conditions for the superiorities of the r-k class predictors over each of these three predictors are obtained by the criterion of mean square error matrix. Furthermore, we suggest tests to approve if these conditions are indeed satisfied. Finally, real data analysis and a simulation study are used to illustrate the findings. © 2018 Elsevier Inc. FDK-2015-3968 This research was supported by Research Fund of Çukurova University under Project Number FDK-2015-3968. We also thank to anonymous referees for their valuable comments and Viktor Witkovsky for his contribution on the simulation codes. Appendix A

We propose the Liu estimator and the Liu predictor via the penalized log-likelihood approach in linear mixed models when multicollinearity is present. The necessary and sufficient conditions for the superiority of the Liu predictor over the best linear unbiased predictor and the ridge predictor of linear combinations of fixed and random effects in the sense of matrix and scalar mean square errors are examined. Furthermore, the selection of the Liu biasing parameter is given and the findings are illustrated with both a real data set and a simulation study. The study show that the Liu estimator and predictor outperform the ridge estimator and predictor and the blue and blup in the sense of mean square error for large degree of correlation and the degree of supremacy of the Liu estimator and predictor over the ridge estimator and predictor and the blue and blup increase as the Liu biasing parameter decreases. © 2018, © 2018 Taylor & Francis Group, LLC.

This article deals with the general form of the hat matrix and the DFBETA measure to detect the influential observations and the leverages in the linear regression model with more than one regressor when the errors are from AR(1) and AR(2) processes. Previous studies dealing with the influential observations and the leverages in the constant mean model and regression through the origin model are obtained as special cases. To demonstrate the utility of the hat matrix and the DFBETA measure, two numerical examples based on the ice cream consumption data with AR(1) errors and the Fox-Hartnagel data with AR(2) errors are analyzed. The results show that the parameter of the autoregressive process affects the influential and leverage points. © 2015 Taylor & Francis Group, LLC.

In this paper, the cross-validation methods namely the $$C_{p}$$ , PRESS and GCV are presented under the multiple linear regression model when multicollinearity exists and additional information imposes restrictions among the parameters that should hold in exact terms. The selection of the biasing parameters are given so as to minimize the cross-validation methods. An example is given which illustrates the comprehensive predictive assessment of various estimators and shows the usefullness of computing. Besides, the performance of the estimators under several different conditions is examined via a simulation study. The results displayed that the biased estimator versions and the restricted form of the biased estimator versions of cross-validation methods give better predictive performance in the presence of multicollinearity.

This paper deals with the problem of multicollinearity in a multiple linear regression model with linear equality restrictions. The restricted two parameter estimator which was proposed in case of multicollinearity satisfies the restrictions. The performance of the restricted two parameter estimator over the restricted least squares (RLS) estimator and the ordinary least squares (OLS) estimator is examined under the mean square error (MSE) matrix criterion when the restrictions are correct and not correct. The necessary and sufficient conditions for the restricted ridge regression, restricted Liu and restricted shrunken estimators, which are the special cases of the restricted two parameter estimator, to have a smaller MSE matrix than the RLS and the OLS estimators are derived when the restrictions hold true and do not hold true. Theoretical results are illustrated with numerical examples based on Webster, Gunst and Mason data and Gorman and Toman data. We conduct a final demonstration of the performance of the estimators by running a Monte Carlo simulation which shows that when the variance of the error term and the correlation between the explanatory variables are large, the restricted two parameter estimator performs better than the RLS estimator and the OLS estimator under the configurations examined. © 2013 © 2013 Taylor & Francis.

The detection of outliers and influential observations has received a great deal of attention in the statistical literature in the context of least-squares (LS) regression. However, the explanatory variables can be correlated with each other and alternatives to LS come out to address outliers/influential observations and multicollinearity, simultaneously. This paper proposes new influence measures based on the affine combination type regression for the detection of influential observations in the linear regression model when multicollinearity exists. Approximate influence measures are also proposed for the affine combination type regression. Since the affine combination type regression includes the ridge, the Liu and the shrunken regressions as special cases, influence measures under the ridge, the Liu and the shrunken regressions are also examined to see the possible effect that multicollinearity can have on the influence of an observation. The Longley data set is given illustrating the influence measures in affine combination type regression and also in ridge, Liu and shrunken regressions so that the performance of different biased regressions on detecting and assessing the influential observations is examined. © 2013 Copyright Taylor and Francis Group, LLC.

WOS: 000398114600022, This article is concerned with the parameter estimation in partly linear regression models when the errors are dependent. To overcome the multicollinearity problem, a generalized Liu estimator is proposed. The theoretical properties of the proposed estimator and its relationship with some existing methods designed for partly linear models are investigated. Finally, a hypothetical data is conducted to illustrate some of the theoretical results.

It is known that collinearity among the explanatory variables in generalized linear models (GLMs) inflates the variance of maximum likelihood estimators. To overcome multicollinearity in GLMs, ordinary ridge estimator and restricted estimator were proposed. In this study, a restricted ridge estimator is introduced by unifying the ordinary ridge estimator and the restricted estimator in GLMs and its mean squared error (MSE) properties are discussed. The MSE comparisons are done in the context of first-order approximated estimators. The results are illustrated by a numerical example and two simulation studies are conducted with Poisson and binomial responses. © 2017 Taylor & Francis Group, LLC