Search

One of the primary issues that arises in statistical modeling pertains to the assessment of the relative importance of each variable in the model. A variety of techniques have been proposed to quantify variable importance for regression models. However, in the context of best subset selection, fewer satisfactory methods are available. With this motivation, we here develop a variable importance measure expressly for this setting. We investigate and illustrate the properties of this measure, introduce algorithms for the efficient computation of its values, and propose a procedure for calculating p-values based on its sampling distributions. We present multiple simulation studies to examine the properties of the proposed methods, along with an application to demonstrate their practical utility. [ABSTRACT FROM AUTHOR]

Sparse regression and feature extraction are the cornerstones of knowledge discovery from massive data. Their goal is to discover interpretable and predictive models that provide simple relationships among scientific variables. While the statistical tools for model discovery are well established in the context of linear regression, their generalization to nonlinear regression in material modeling is highly problem‐specific and insufficiently understood. Here we explore the potential of neural networks for automatic model discovery and induce sparsity by a hybrid approach that combines two strategies: regularization and physical constraints. We integrate the concept of Lp regularization for subset selection with constitutive neural networks that leverage our domain knowledge in kinematics and thermodynamics. We train our networks with both, synthetic and real data, and perform several thousand discovery runs to infer common guidelines and trends: L2 regularization or ridge regression is unsuitable for model discovery; L1 regularization or lasso promotes sparsity, but induces strong bias that may aggressively change the results; only L0 regularization allows us to transparently fine‐tune the trade‐off between interpretability and predictability, simplicity and accuracy, and bias and variance. With these insights, we demonstrate that Lp regularized constitutive neural networks can simultaneously discover both, interpretable models and physically meaningful parameters. We anticipate that our findings will generalize to alternative discovery techniques such as sparse and symbolic regression, and to other domains such as biology, chemistry, or medicine. Our ability to automatically discover material models from data could have tremendous applications in generative material design and open new opportunities to manipulate matter, alter properties of existing materials, and discover new materials with user‐defined properties. [ABSTRACT FROM AUTHOR]

Background: Difficult airway management poses a high-risk procedure, potentially leading to patient mortality. Despite significant advancements in modern anesthesia over the years, there remains a lack of a single reliable tool for predicting difficult airways. Objectives: This study aims to examine neck fat diameter as a potential predictor of airway difficulty. Methods: Two hundred fifty patients participated in the study. Airway parameters, including Mallampati score, mandibular protrusion, upper lip bite test, thyromental distance, neck movements, and neck fat diameter, were documented. Results: Among our study population, 48 patients experienced difficult intubation. The Mann-Whitney test revealed that submental fat was significantly greater in the difficult intubation group (P-value < 0.001). Based on the best subset variable selection, submental fat, thyromental distance (model 1), and age (model 2) were selected to participate in the multivariate regression model. Each one mm increase in neck fat was associated with a 30% increase in the odds of difficult intubation (Pvalue < 0.001) in the univariate logistic model. After adjusting for thyromental distance, age, and gender, the odds of difficult intubation were approximately 2 times higher in patients with 1 mm more neck fat (P-value 0.007). The area under the curve of the Receiver Operating Characteristics curve and McFadden's pseudo-R squared of the multivariate model were 0.998 and 0.91, respectively. Conclusions: Neck fat diameter serves as a reliable and straightforward predictor of airway difficulty and can be used alone or in conjunction with other predictors for the prediction of difficult airway management. [ABSTRACT FROM AUTHOR]

Typically, in high dimensional data sets, many covariates are not significantly associated with a response. Moreover, those covariates are highly correlated, leading to a multicollinearity problem. Hence, the model is sparse since the coefficient of most covariates are likely to be zero. The classical frequentist or likelihood-based variable selection via any criterion such as Bayesian Information Criteria (BIC) and Akaike Information Criteria (AIC) or a stepwise subset selection becomes infeasible when the number of variables are large. An alternative solution is a Bayesian variable selection. In this study, we used a variable selection via a Bayesian variable selection and the least absolute shrinkage and selection operator (LASSO) method in the logistic regression model. Moreover, those methods were expanded to be applied to real datasets. [ABSTRACT FROM AUTHOR]

We consider the stochastic contextual bandit problem under the high dimensional linear model. We focus on the case where the action space is finite and random, with each action associated with a randomly generated contextual covariate. This setting finds essential applications such as personalized recommendations, online advertisements, and personalized medicine. However, it is very challenging to balance the exploration and exploitation tradeoff. We modify the LinUCB algorithm in doubly growing epochs and estimate the parameter using the best subset selection method, which is easy to implement in practice. This approach achieves O (s T) regret with high probability, which is nearly independent of the "ambient" regression model dimension d. We further attain a sharper O (s T) regret by using the SupLinUCB framework and match the minimax lower bound of the low-dimensional linear stochastic bandit problem. Finally, we conduct extensive numerical experiments to empirically demonstrate our algorithms' applicability and robustness. for this article are available online. [ABSTRACT FROM AUTHOR]

We introduce a generalized additive model for location, scale, and shape (GAMLSS) next of kin aiming at distribution-free and parsimonious regression modeling for arbitrary outcomes. We replace the strict parametric distribution formulating such a model by a transformation function, which in turn is estimated from data. Doing so not only makes the model distribution-free but also allows to limit the number of linear or smooth model terms to a pair of location-scale predictor functions. We derive the likelihood for continuous, discrete, and randomly censored observations, along with corresponding score functions. A plethora of existing algorithms is leveraged for model estimation, including constrained maximum-likelihood, the original GAMLSS algorithm, and transformation trees. Parameter interpretability in the resulting models is closely connected to model selection. We propose the application of a novel best subset selection procedure to achieve especially simple ways of interpretation. All techniques are motivated and illustrated by a collection of applications from different domains, including crossing and partial proportional hazards, complex count regression, nonlinear ordinal regression, and growth curves. All analyses are reproducible with the help of the tram add-on package to the R system for statistical computing and graphics. [ABSTRACT FROM AUTHOR]

Linear mixed models (LMMs) are instrumental for regression analysis with structured dependence, such as grouped, clustered, or multilevel data. However, selection among the covariates—while accounting for this structured dependence—remains a challenge. We introduce a Bayesian decision analysis for subset selection with LMMs. Using a Mahalanobis loss function that incorporates the structured dependence, we derive optimal linear coefficients for (i) any given subset of variables and (ii) all subsets of variables that satisfy a cardinality constraint. Crucially, these estimates inherit shrinkage or regularization and uncertainty quantification from the underlying Bayesian model, and apply for any well‐specified Bayesian LMM. More broadly, our decision analysis strategy deemphasizes the role of a single "best" subset, which is often unstable and limited in its information content, and instead favors a collection of near‐optimal subsets. This collection is summarized by key member subsets and variable‐specific importance metrics. Customized subset search and out‐of‐sample approximation algorithms are provided for more scalable computing. These tools are applied to simulated data and a longitudinal physical activity dataset, and demonstrate excellent prediction, estimation, and selection ability. [ABSTRACT FROM AUTHOR]

Simplicity, efficiency and precision are basic principles for modeling and analyzing of coal-fired boilers data. However, the load fluctuations, system delay, multi-variable coupling pose great challenges to the high-precision modeling for NOx emission. A hybrid feature selection method for boiler data is proposed to predict accurately NOx emission. Firstly, the mechanism analysis is used to narrow the feature scope, and the feature set is selected preliminarily. Secondly, maximum information coefficient (MIC) method is introduced to calculate the correlation between features and NOx information to eliminate boiler system delay. Thirdly, a combined feature evaluation method is developed, which integrates Filter and Embedded method to obtain feature ranking, then the ranking information is regarded as priori knowledge to improve the genetic algorithm. Finally, a fitness function to maximize prediction accuracy and minimize feature dimension is constructed based on hybrid method to realize feature subset selection and NOx emission prediction. Original data are collected from one 1000 MW coal-fired unit in the Guangdong province of China. Experimental results show that the number of features is reduced by nearly 70%, the MAPE of LightGBM regression model is no more than 2%. Higher prediction accuracy can be obtained using the features extracted through the proposed hybrid method. [ABSTRACT FROM AUTHOR]

A bottleneck in Laser Powder Bed Fusion (L-PBF) metal additive manufacturing (AM) is the quality inconsistency of its products. To address this issue without costly experimentation, computational multi-physics modeling has been used, but the effectiveness is limited by parameter uncertainties and their interactions. We propose a full factorial design and variable selection approach for the analytics of main and interaction effects arising from material parameter uncertainties in multi-physics models. Data is collected from high-fidelity thermal-fluid simulations based on a 2-level full factorial design for 5 selected material parameters. Crucial physical phenomena of the L-PBF process are analyzed to extract physics-based domain knowledge, which are used to establish a validation checkpoint for our study. Initial data visualization with half-normal probability plots, interaction plots and standard deviation plots, is used to assess if the checkpoint is being met. We then apply the combination of best subset selection and the LASSO method on multiple linear regression models for comprehensive variable selection. Analytics yield statistically and phyiscally validated findings with practical implications, emphasizing the importance of parameter interactions under uncertainty, and their relation to the underlying physics of L-PBF. [ABSTRACT FROM AUTHOR]

In low-dimensional data and within the framework of a classical linear regression model, we intend to compare variable selection methods and investigate the role of shrinkage of regression estimates in a simulation study. Our primary aim is to build descriptive models that capture the data structure parsimoniously, while our secondary aim is to derive a prediction model. Simulation studies are an important tool in statistical methodology research if they are well designed, executed, and reported. However, bias in favor of an "own" preferred method is prevalent in most simulation studies in which a new method is proposed and compared with existing methods. To overcome such bias, neutral comparison studies, which disregard the superiority or inferiority of a particular method, have been proposed. In this paper, we designed a simulation study with key principles of neutral comparison studies in mind, though certain unintentional biases cannot be ruled out. To improve the design and reporting of a simulation study, we followed the recently proposed ADEMP structure, which entails defining the aims (A), data-generating mechanisms (D), estimand/target of analysis (E), methods (M), and performance measures (P). To ensure the reproducibility of results, we published the protocol before conducting the study. In addition, we presented earlier versions of the design to several experts whose feedback influenced certain aspects of the design. We will compare popular penalized regression methods (lasso, adaptive lasso, relaxed lasso, and nonnegative garrote) that combine variable selection and shrinkage with classical variable selection methods (best subset selection and backward elimination) with and without post-estimation shrinkage of parameter estimates. [ABSTRACT FROM AUTHOR]

As a tool for producing meaningful and interpretable results, subset or variable selection has been well studied in modern statistics. However, most of the existing methods focus on the independent data and cannot directly extend to the network-linked data where samples are connected with each other. To this end, we propose a subset selection method in the linear regression model by incorporating the network information into the intercept term, which can achieve automatic subset selection and have good network structural interpretability simultaneously. Based on this, we develop an efficient algorithm to recover the true subset, as well as determine subgroups. Simulation studies demonstrate that the proposal outperforms the state-of-art methods in estimation and selection accuracy. We also apply the proposed method on data from the national longitudinal study of adolescent health and show the superiority of selecting variables alone a network by a smaller model size and more accurate prediction. [ABSTRACT FROM AUTHOR]

Methods for reducing distributed subsample sizes have increasingly become popular statistical problems in the big data era. Existing works of optimal subsample selection on the massive linear and generalized linear models with distributed data sources have been solidly investigated and widely applied. Nevertheless, few studies have developed distributed optimal subsample selection procedures for quantile regression in massive data. In such settings, the distributed optimal subsampling probabilities and subset sizes selection criteria need to be established simultaneously. In this work, we propose a distributed subsampling technique for the quantile regression models. The estimation approach is based on a two-step algorithm for the distributed subsampling procedures. Furthermore, the theoretical results, such as consistency and asymptotic normality of resultant estimators, are rigorously established under some regularity conditions. The empirical evaluation and performance of the proposed subsampling method are conducted in simulation experiments and real data applications. • We develop a distributed optimal subsampling method for quantile regression. • The asymptotic properties of the proposed algorithms are rigorously established. • We provide a distributed least squares resampling covariance estimation method. [ABSTRACT FROM AUTHOR]

Water quality indices (WQIs) are used for the simple assessment and classification of the water quality of surface water sources. However, considerable time, financial resources, and effort are required to measure the parameters used for their calculation. Prediction of WQIs through supervised machine learning is a useful and simple approach to reduce the cost of the analysis through the development of predictive models with a reduced number of water quality parameters. In this study, regression and classification machine-learning models were developed to estimate the ecosystem-specific WQI previously developed for the Santiago-Guadalajara River (SGR-WQI), which involves the measurement of 17 water quality parameters. The best subset selection method was employed to reduce the number of significant parameters required for the SGR-WQI prediction. The multiple linear regression model using 12 parameters displayed a residual square error (RSE) of 3.262, similar to that of the multiple linear regression model using 17 parameters (RSE = 3.255), which translates into significant savings for WQI estimation. Additionally, the generalized additive model not only displayed an adjusted R2 of 0.9992, which is the best fit of all the models evaluated, but also fitted the rating curves of each parameter developed for the original algorithm for the SGR-WQI calculation with great accuracy. Regarding the classification models, an overall proportion of 93% and 86% of data were correctly classified using the logistic regression model with 17 and 12 parameters, respectively, while the linear discriminant functions using 12 parameters correctly classified an overall proportion of 84%. The models evaluated were found to be efficient in predicting the SGR-WQI with a reduced number of parameters as complementary tools to extend the current water quality monitoring program of the Santiago-Guadalajara River. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new approach for the simulation-based support of tryout operations in deep drawing which can be schematically classified as automatic knowledge acquisition. The central idea is to identify information maximising sensor positions for draw-in as well as local blank holder force sensors by solving the column subset selection problem with respect to the sensor sensitivities. Inverse surrogate models are then trained using the selected sensor signals as predictors and the material and process parameters as targets. The final models are able to observe the drawing process by estimating current material and process parameters, which can then be compared to the target values to identify process corrections. The methodology is examined on an Audi A8L side panel frame using a set of 635 simulations, where 20 out of 21 material and process parameters can be estimated with an R2 value greater than 0.9. The result shows that the observational models are not only capable of estimating all but one process parameters with high accuracy, but also allow the determination of material parameters at the same time. Since no assumptions are made about the type of process, sensors, material or process parameters, the methodology proposed can also be applied to other manufacturing processes and use cases. [ABSTRACT FROM AUTHOR]

Over the past decades, variable selection for high-dimensional data has drawn increasing attention. With a large number of predictors, there rises a big challenge for model fitting and prediction. In this paper, we develop a new Bayesian method of best subset selection using a hybrid search algorithm that combines a deterministic local search and a stochastic global search. To reduce the computational cost of evaluating multiple candidate subsets for each update, we propose a novel strategy that enables us to calculate exact marginal likelihoods of all neighbor models simultaneously in a single computation. In addition, we establish model selection consistency for the proposed method in the high-dimensional setting in which the number of possible predictors can increase faster than the sample size. Simulation study and real data analysis are conducted to investigate the performance of the proposed method. [ABSTRACT FROM AUTHOR]

Subset selection in multiple linear regression is to identify a subset of explanatory variables that are most relevant to the response variable while discarding the remaining variables. Traditionally, a regression model is obtained by solving a given optimization problem, and then, some statistical tests and evaluations are performed to confirm its validity and to verify the satisfaction of qualifying requirements. Practically, human intervention is necessary at this stage. To prevent the need for manual inspection, we propose a new mixed integer programming formulation for the multiple linear regression subset selection problem. Our model enforces the statistical significance and the multicollinearity through explicit constraints along with several other well-known criteria. In contrast to existing approaches that incorporate these criteria gradually through a cutting-plane structure, this effort considers them explicitly within the formulation. The effectiveness of the approach is evaluated over different real datasets and compared with the existing methods in the literature. The results demonstrate that the significance of coefficients can be provided without any considerable change in the value of mean squared error. [ABSTRACT FROM AUTHOR]

The abundance of available digital big data has created new challenges in identifying relevant variables for regression models. One statistical problem that gained relevance in the era of big data is high-dimensional statistical inference, when the number of variables greatly exceeds the number of observations. Typically, prediction errors in linear regression skyrocket when the number of included variables gets close to the number of observations, and ordinary least squares (OLS) regression no longer works in a high-dimensional scenario. Regularized estimators as a feasible solution include the Least Absolute Shrinkage and Selection Operator (Lasso), which we introduce to communication scholars here. We will include the statistical background of this technique that combines estimation and variable selection simultaneously and helps identify relevant variables for regression models in high-dimensional scenarios. We contrast the Lasso with two alternative strategies of selecting variables for regression models, namely, a theory-based "subset selection" of variables and a nonselective "all in" strategy. The simulation shows that the Lasso produces lower and more relatively stable prediction errors than the two alternative variable selection strategies, and it is therefore recommended to use, especially in high-dimensional settings typical in times of big data analysis. [ABSTRACT FROM AUTHOR]

Subset selection in multiple linear regression aims to choose a subset of candidate explanatory variables that tradeoff fitting error (explanatory power) and model complexity (number of variables selected). We build mathematical programming models for regression subset selection based on mean square and absolute errors, and minimal-redundancy–maximal-relevance criteria. The proposed models are tested using a linear-program-based branch-and-bound algorithm with tailored valid inequalities and big M values and are compared against the algorithms in the literature. For high dimensional cases, an iterative heuristic algorithm is proposed based on the mathematical programming models and a core set concept, and a randomized version of the algorithm is derived to guarantee convergence to the global optimum. From the computational experiments, we find that our models quickly find a quality solution while the rest of the time is spent to prove optimality; the iterative algorithms find solutions in a relatively short time and are competitive compared to state-of-the-art algorithms; using ad-hoc big M values is not recommended. [ABSTRACT FROM AUTHOR]

Objective: To explore the frequency and factors leading to cyanotic and acyanotic congenital heart diseases in children Methodology: A sample of 430 children with congenital heart diseases were enrolled in the study from the Pediatric Cardiology Department at CPE Institute of Cardiology Multan, Pakistan. Data regarding different factors were collected through direct personal interviews form mothers and the children files maintained in the hospital. Responses were noted on Proforma. For Statistical analysis, we adopted the bivariate frequency analysis and Logistic regression model based on the best subset selection. Odds ratios and a 95% confidence interval were also calculated to measure the contribution of risk factors. Results: Out of 430 children, the male-female ratio in Cyanotic and acyanotic congenital heart diseases is 2.5:1 and 1.3:1 respectively. The 54.6% and 45.8% of mothers of cyanotic children and acyanotic children respectively are taken poor nutrition during their pregnancy. Conclusion: Improving the physical activity, nutrition, partner interaction, access to basic health care facilities, calories in food, environment, and housing condition during the pregnancy period reduce the risk factors of cyanotic and acyanotic congenital heart diseases in children. [ABSTRACT FROM AUTHOR]

In this paper, we propose two estimating equation-based methods for estimating the regression parameter vector in a multiplicative regression model when a subset of covariates is subject to a measurement error, but replicate measurements of their surrogates are available. Both methods allow the number of replicate mea- surements to vary between subjects. No parametric assumption is imposed on the measurement error term or the true covariates, which are not observed in the data set. Under some regularity conditions, the asymptotic normality is proved for both proposed estimators. Furthermore, the estimators are compared theoretically when the distribution of the measurement error follows a normal distribution. Simula- tion studies are conducted to assess the performance of the proposed methods. A real-data analysis is used to illustrate our methods. [ABSTRACT FROM AUTHOR]

In this paper we consider two relevant optimization problems: the problem of selecting the best sparse linear regression model and the problem of optimally identifying the parameters of auto-regressive models based on time series data. Usually these problems, which although different are indeed related, are solved through a sequence of separate steps, alternating between choosing a subset of features and then finding a best fit regression. In this paper we propose to model both problems as mixed integer non linear optimization ones and propose numerical procedures based on state of the art optimization tools in order to solve both of them. The proposed approach has the advantage of considering both model selection as well as parameter estimation as a single optimization problem. Numerical experiments performed on widely available datasets as well as on synthetic ones confirm the high quality of our approach, both in terms of the quality of the resulting models and in terms of CPU time. [ABSTRACT FROM AUTHOR]

A significant issue that has plagued data-driven Vibration-based Structural Health Monitoring (VSHM) is the mitigation of Environmental and Operational Variations (EOVs). The Damage Sensitive Features (DSFs) that are obtained from the vibration response of the structure are influenced by EOVs. Regression analysis, such as multivariate nonlinear regression, can be used to create relationships between Environmental and Operational Parameters (EOPs) and the DSFs, with EOV-insensitive DSFs created by taking the regression residuals. Inherent issues, originating from nuances in their design, exist within the design of the regression models, following from the overall uncertainty and redundancy in predictors and explained variables, leading to poor performance. To overcome this, a comprehensive nonlinear stepwise regression methodology has been developed to scour the regression models of as much uncertainty as possible. The proposed methodology addresses a number of crucial ideas: removing co-nonlinear variables, identifying the most influential EOPs, facilitating the selection of compact regression bases and determining which DSFs should be regressed. Robust DSFs are created by combining non-regressed DSFs with critically thought-out regressed DSFs. Ultimately, reducing the uncertainty within the models will lead to more confidence in the decision making within a VSHM methodology. • Long-term monitoring of an operational wind turbine blade with artificial damage. • Mitigation of environmental and operational variations for robust damage detection. • Nonlinear forward stepwise method for automatic subset and model order selection. • Informed combination of regressed and non-regressed damage sensitive features. [ABSTRACT FROM AUTHOR]

Background: The 24-h area under the concentration-time curve (AUC24)/minimal inhibitory concentration ratio is the best predictive pharmacokinetic/pharmacodynamic (PK/PD) parameter of the efficacy of first-line anti-tuberculosis (TB) drugs. An optimal sampling strategy (OSS) is useful for accurately estimating AUC24; however, OSS has not been developed in the fed state or in the early phase of treatment for first-line anti-TB drugs.Methods: An OSS for the prediction of AUC24 of isoniazid, rifampicin, ethambutol and pyrazinamide was developed for TB patients starting treatment. A prospective, randomized, crossover trial was performed during the first 3 days of treatment in which first-line anti-TB drugs were administered either intravenously or in fasting or fed conditions. The PK data were used to develop OSS with best subset selection multiple linear regression. The OSS was internally validated using a jackknife analysis and externally validated with other patients from different ethnicities and in a steady state of treatment.Results: OSS using time points of 2, 4 and 8 h post-dose performed best. Bias was < 5% and imprecision was < 15% for all drugs except ethambutol in the fed condition. External validation showed that OSS2-4-8 cannot be used for rifampicin in steady state conditions.Conclusion: OSS at 2, 4 and 8 h post-dose enabled an accurate and precise prediction of AUC24 values of first-line anti-TB drugs in this population.Trial Registration: ClinicalTrials.gov (NCT02121314). [ABSTRACT FROM AUTHOR]

In the financial industry, identifying useful information from big data becomes a key research topic. Since the vast number of technical indicators can be captured nowadays, the indicator selection can be used to support investment decision for different financial products concurrently; however, this process is still required experience from investors. In this article, we propose a novel recommendation system which is incorporated with technical indicators. The method of fuzzy subset selection is used to feature relevant indicators which have more impact to the variation in the transaction history. The proposed method enables automatic customization of indicators for different financial products in different markets. In particular, the least absolute distance fuzzy regression with non-symmetric lower and upper bounds is proposed to avoid extreme values in dominating the model. Furthermore, to reduce computational complexity in the subset selection, the selection algorithm operates in the frequency domain for identifying and matching key patterns and peaks in transacted volumes with the technical indicators. This method performs very effective although the number of factors is much greater than the sample size. The proposed method can benefit participants in the finance markets to customize their own trading dashboard as well as set up their own trading strategies. [ABSTRACT FROM AUTHOR]

• Improved method in a class of ill-conditioned parameter estimation problems. • A subset selection criterion based on MSE for the estimates. • Statistical analysis to compare the proposed and conventional methods. • Performance comparison through two case studies: linear and nonlinear regressions. • The proposed method gives better estimates with smaller variances. In cases of ill-conditioned estimation problems, not all parameters can be estimated accurately and a selection of parameter subset composed of more influential and less correlated parameters may be needed to increase parameter estimability. This paper proposes to choose a subset for estimation from transformed parameters along the directions of the principal components of the parameter covariance matrix while retaining the initial guesses for the unselected transformed parameters. Since estimating a subset of the transformed parameters can adjust all the original parameter values and any constraint on the original parameter values can still be applied to the transformed parameters, the proposed regularization method can overcome the limitation of the existing methods, i.e., parameter subset selection and truncated singular value decomposition (also known as 'principal component regression'). It is demonstrated that the proposed method can provide better parameter estimates with smaller variances than the existing parameter subset selection methods, first through statistical analysis, and then through case studies of linear and nonlinear regressions. In addition, based on the derived statistical properties, a criterion is suggested for selecting an optimal subset, which gives the smallest mean squared error of the estimates. Furthermore, with the advantage of a lower variance of the estimates, the proposed regularization method gives a more consistent choice of the number of parameters to estimate with the smallest mean squared error of the estimates even when significant errors in the initial guesses and high levels of measurement noise exist. [ABSTRACT FROM AUTHOR]

In this paper, we propose the hard thresholding regression (HTR) for estimating high‐dimensional sparse linear regression models. HTR uses a two‐stage convex algorithm to approximate the ℓ0‐penalized regression: The first stage calculates a coarse initial estimator, and the second stage identifies the oracle estimator by borrowing information from the first one. Theoretically, the HTR estimator achieves the strong oracle property over a wide range of regularization parameters. Numerical examples and a real data example lend further support to our proposed methodology. [ABSTRACT FROM AUTHOR]

In this research, Bayesian model averaging (BMA) and least absolute shrinkage and selection operator regression (LASSO) algorithms were adopted for the core permeability modeling as a function of well log and core measurements. More specifically, the core permeability (dependent factor) was modeled given the well log and core data (independent variables) and then predicted in non-cored intervals of a well in a sandstone formation. The BMA is a stochastic linear modeling and a Bayesian parameter selection. Among 50 linear models generated as a function of the independent variables and depicted in Occam's window, the best model of the highest posterior probability is determined. The best model has the optimal subset parameters that most influences the response factor. The core permeability modeling was again conducted using the LASSO algorithm, which adopts a completely different way of modeling and subset selection by using the penalized least-squared equation. Both BMA and LASSO resulted in a very accurate prediction of the core permeability by achieving perfect measured and calculated permeability matching. Matches for the two algorithms were quantified by computing the adjusted R-squared and root-mean-squared prediction error. In addition, results of the BMA and LASSO were compared to the conventional multiple linear regression (LM). The LASSO algorithm led to an accurate matching similar to LM, but slightly better than BMA. Therefore, both BMA and LASSO represent integrated procedures for accurate permeability estimation similar to conventional regression analysis. [ABSTRACT FROM AUTHOR]

The multi-target regression problem comprises the prediction of multiple continuous variables at the same time using a common set of input variables, and in the last few years, this problem has gained an increasing attention due to the broad range of real-world applications that can be analyzed under this framework. The complexity of the multi-target regression problem is higher than the single-target regression one since target variables often have statistical dependencies, and these dependencies should be correctly exploited in order to effectively solve this problem. Consequently, additional difficulties appear when the aim is to perform a selection of instances on this type of data. In this work, an ensemble-based method to perform the instance selection task in multi-target regression problems is proposed. First, a well-known instance selection method is adapted to directly work with multi-target data. Second, the proposed ensemble-based approach uses a set of these adapted methods to select the final subset of instances. The members of the ensemble select partial data subsets, where each member is performed on a different input space that is expanded with target variables, exploiting therefore the underlying inter-target dependencies. Finally, the ensemble-based method aggregates all the selected partial data subsets into a final subset of relevant instances by means of solving an optimization problem with a simple greedy heuristic. The experimental study carried out on 18 datasets shows the effectiveness of our proposal for selecting instances in the multi-target regression problem. Results demonstrate that the size of datasets is considerably reduced, whilst the predictive performance of the multi-target regressors is maintained or even improved. Also, it is observed that the proposed method is robust to the presence of noise in data. [ABSTRACT FROM AUTHOR]

Statistical model selection is a great challenge when the number of accessible measurements is much smaller than the dimension of the parameter space. We study the problem of model selection in the context of subset selection for high-dimensional linear regressions. Accordingly, we propose a new model selection criterion with the Fisher information that leads to the selection of a parsimonious model from all the combinatorial models up to some maximum level of sparsity. We analyze the performance of our criterion as the number of measurements grows to infinity, as well as when the noise variance tends to zero. In each case, we prove that our proposed criterion gives the true model with a probability approaching one. Additionally, we devise a computationally affordable algorithm to conduct model selection with the proposed criterion in practice. Interestingly, as a side product, our algorithm can provide the ideal regularization parameter for the Lasso estimator such that Lasso selects the true variables. Finally, numerical simulations are included to support our theoretical findings. [ABSTRACT FROM AUTHOR]

The subset polynomial regression model is wider than the polynomial regression model. This study proposes an estimate of the parameters of the subset polynomial regression model with unknown error and distribution. The Bootstrap method is used to estimate the parameters of the subset polynomial regression model. Simulated data is used to test the performance of the Bootstrap method. The test results show that the bootstrap method can estimate well the parameters of the subset polynomial regression model. [ABSTRACT FROM AUTHOR]

The Akaike information criterion (AIC) is routinely used for model selection in best subset regression. The standard AIC, however, generally under-penalizes model complexity in the best subset regression setting, potentially leading to grossly overfit models. Recently, Zhang and Cavanaugh (Comput Stat 31(2):643-669, 2015) made significant progress towards addressing this problem by introducing an effective multistage model selection procedure. In this paper, we present a rigorous and coherent conceptual framework for extending AIC to best subset regression. A new model selection algorithm derived from our framework possesses well understood and desirable asymptotic properties and consistently outperforms the procedure of Zhang and Cavanaugh in simulation studies. It provides an effective tool for combating the pervasive overfitting that detrimentally impacts best subset regression analysis so that the selected models contain fewer irrelevant predictors and predict future observations more accurately. [ABSTRACT FROM AUTHOR]

High signal to noise ratio (SNR) consistency of model selection criteria in linear regression models has attracted a lot of attention recently. However, most of the existing literature on high SNR consistency deals with model order selection. Further, the limited literature available on the high SNR consistency of subset selection procedures (SSPs) is applicable to linear regression with full rank measurement matrices only. Hence, the performance of SSPs used in under determined linear models (a.k.a compressive sensing (CS) algorithms) at high SNR is largely unknown. This paper fills this gap by deriving necessary and sufficient conditions for the high SNR consistency of popular CS algorithms like l 0 -minimization, basis pursuit de-noising or LASSO, orthogonal matching pursuit and Dantzig selector. Necessary conditions analytically establish the high SNR inconsistency of CS algorithms when used with the tuning parameters discussed in literature. Novel tuning parameters with SNR adaptations are developed using the sufficient conditions and the choice of SNR adaptations are discussed analytically using convergence rate analysis. CS algorithms with the proposed tuning parameters are numerically shown to be high SNR consistent and outperform existing tuning parameters in the moderate to high SNR regime. [ABSTRACT FROM AUTHOR]

Adjacent category logit models are ordered regression models that focus on comparisons of adjacent categories. These models are particularly useful for ordinal response variables with categories that are of substantive interest. In this article, we consider unconstrained and constrained versions of the partial adjacent category logit model, which is an extension of the traditional model that relaxes the proportional odds assumption for a subset of independent variables. In the unconstrained partial model, the variables without proportional odds have coefficients that freely vary across cutpoint equations, whereas in the constrained partial model two or more of these variables have coefficients that vary by common factors. We improve upon an earlier formulation of the constrained partial adjacent category model by introducing a new estimation method and conceptual justification for the model. Additionally, we discuss the connections between partial adjacent category models and other models within the adjacent approach, including stereotype logit and multinomial logit. We show that the constrained and unconstrained partial models differ only in terms of the number of dimensions required to describe the effects of variables with nonproportional odds. Finally, we illustrate the partial adjacent category logit models with empirical examples using data from the international social survey program and the general social survey. [ABSTRACT FROM AUTHOR]

In this paper, we translate variable selection for linear regression into multiple testing, and select significant variables according to testing result. New variable selection procedures are proposed based on the optimal discovery procedure (ODP) in multiple testing. Due to ODP’s optimality, if we guarantee the number of significant variables included, it will include less non significant variables than marginalp-value based methods. Consistency of our procedures is obtained in theory and simulation. Simulation results suggest that procedures based on multiple testing have improvement over procedures based on selection criteria, and our new procedures have better performance than marginalp-value based procedures. [ABSTRACT FROM AUTHOR]

Regression models with a large number of predictors arise in diverse fields of social sciences and natural sciences. For proper interpretation, we often would like to identify a smaller subset of the variables that shows the strongest information. In such a large size of candidate predictors setting, one would encounter a computationally cumbersome search in practice by optimizing some criteria for selecting variables, such as AIC, $$C_{P}$$ and BIC, through all possible subsets. In this paper, we present two efficient optimization algorithms vis Markov chain Monte Carlo (MCMC) approach for searching the global optimal subset. Simulated examples as well as one real data set exhibit that our proposed MCMC algorithms did find better solutions than other popular search methods in terms of minimizing a given criterion. [ABSTRACT FROM AUTHOR]

We describe a method for estimating a probability density function when some of the sufficient statistics are known. The general form of the PDF is within the exponential family and is known as the exponentially embedded family. Using the proposed estimator new approaches to choosing features from a set of possible features become available, with the error metric being the Kullback-Liebler distance. Applications to subset selection in the context of multipath estimation as well as linear regression for machine learning are used to illustrate the practical utility of the method. [ABSTRACT FROM PUBLISHER]

Stability selection (multisplit) approach is a variable selection procedure which relies on multisplit data to overcome the shortcomings that may occur to single-split data. Unfortunately, this procedure yields very poor results in the presence of outliers and other contamination in the original data. The problem becomes more complicated when the regression residuals are serially correlated. This paper presents a new robust stability selection procedure to remedy the combined problem of autocorrelation and outliers. We demonstrate the good performance of our proposed robust selection method using real air quality data and simulation study. [ABSTRACT FROM AUTHOR]

This paper describes the R package VSURF. Based on random forests, and for both regression and classification problems, it returns two subsets of variables. The first is a subset of important variables including some redundancy which can be relevant for interpretation, and the second one is a smaller subset corresponding to a model trying to avoid redundancy focusing more closely on the prediction objective. The two-stage strategy is based on a preliminary ranking of the explanatory variables using the random forests permutation-based score of importance and proceeds using a stepwise forward strategy for variable introduction. The two proposals can be obtained automatically using data-driven default values, good enough to provide interesting results, but strategy can also be tuned by the user. The algorithm is illustrated on a simulated example and its applications to real datasets are presented. [ABSTRACT FROM AUTHOR]

A predictive quantitative structure–property relationships (QSPR) is developed for modeling the retention index measured on the OV-101 glass capillary gas chromatography column, in a set of 1208 flavor and fragrance compounds. The 4885 molecular descriptors are calculated using the Dragon software and then are simultaneously analyzed through multivariable linear regression analysis using the replacement method (RM) variable subset selection technique. We proceed in three steps, the first one by considering all descriptor blocks, the second one by excluding conformational descriptors blocks, and the last one by analyzing only 3D-descriptors families. The models are properly validated through an external test set of compounds. Cross-validation methods such as leave-one-out and leave-more-out are applied, together with Y-randomization and applicability domain analysis. The results clearly suggest that 3D-descriptors do not offer relevant information for modeling the retention index, while a topological index such as the solvation connectivity index of first order has a high relevance for this purpose. [ABSTRACT FROM AUTHOR]

This paper concerns a method of selecting the best subset of explanatory variables for a linear regression model. Employing Mallows’ C p as a goodness-of-fit measure, we formulate the subset selection problem as a mixed integer quadratic programming problem. Computational results demonstrate that our method provides the best subset of variables in a few seconds when the number of candidate explanatory variables is less than 30. Furthermore, when handling datasets consisting of a large number of samples, it finds better-quality solutions faster than stepwise regression methods do. [ABSTRACT FROM AUTHOR]

We consider in this paper quadratic programming problems with cardinality and minimum threshold constraints that arise naturally in various real-world applications such as portfolio selection and subset selection in regression. This class of problems can be formulated as mixed-integer 0-1 quadratic programs. We propose a new semidefinite program (SDP) approach for computing the "best" diagonal decomposition that gives the tightest continuous relaxation of the perspective reformulation of the problem. We also give an alternative way of deriving the perspective reformulation by applying a special Lagrangian decomposition scheme to the diagonal decomposition of the problem. This derivation can be viewed as a "dual" method to the convexification method employing the perspective function on semicontinuous variables. Computational results show that the proposed SDP approach can be advantageous for improving the performance of mixed-integer quadratic programming solvers when applied to the perspective reformulations of the problem. [ABSTRACT FROM AUTHOR]

A column generation based approach is proposed for solving the cluster-wise regression problem. The proposed strategy relies firstly on several efficient heuristic strategies to insert columns into the restricted master problem. If these heuristics fail to identify an improving column, an exhaustive search is performed starting with incrementally larger ending subsets, all the while iteratively performing heuristic optimization to ensure a proper balance of exact and heuristic optimization. Additionally, observations are sequenced by their dual variables and by their inclusion in joint pair branching rules. The proposed strategy is shown to outperform the best known alternative (BBHSE) when the number of clusters is greater than three. Additionally, the current work further demonstrates and expands the successful use of the new paradigm of using incrementally larger ending subsets to strengthen the lower bounds of a branch and bound search as pioneered by Brusco's Repetitive Branch and Bound Algorithm (RBBA). [ABSTRACT FROM AUTHOR]

There is a rich literature on Bayesian variable selection for parametric models. Our focus is on generalizing methods and asymptotic theory established for mixtures ofg-priors to semiparametric linear regression models having unknown residual densities. Using a Dirichlet process location mixture for the residual density, we propose a semiparametricg-prior which incorporates an unknown matrix of cluster allocation indicators. For this class of priors, posterior computation can proceed via a straightforward stochastic search variable selection algorithm. In addition, Bayes’ factor and variable selection consistency is shown to result under a class of proper priors ongeven when the number of candidate predictorspis allowed to increase much faster than sample sizen, while making sparsity assumptions on the true model size. [ABSTRACT FROM AUTHOR]

For the additive white Gaussian noise channel with average codeword power constraint, sparse superposition codes are developed. These codes are based on the statistical high-dimensional regression framework. In a previous paper, we investigated decoding using the optimal maximum-likelihood decoding scheme. Here, a fast decoding algorithm, called the adaptive successive decoder, is developed. For any rate R less than the capacity \cal C, communication is shown to be reliable with nearly exponentially small error probability. Specifically, for blocklength n, it is shown that the error probability is exponentially small in n/\log n. [ABSTRACT FROM AUTHOR]

This paper first reduces the problem of detecting structural breaks in a random walk to that of finding the best subset of explanatory variables in a regression model and then tailors various subset selection criteria to this specific problem. Of particular interest are those new criteria, which are obtained by means of simulation using the efficient algorithm of Bai and Perron (J Appl Econom 18:1–22, 2003 ). Unlike conventional variable selection methods, which penalize new variables entering a model either in the same way (e.g., AIC and BIC) or milder (e.g., MRIC and $$\mathrm {FPE}_\mathrm{{sub}}$$ ) than already included variables, they do not follow any monotonic penalizing scheme. In general, their non-monotonicity is more pronounced in the case of fat tails. The characteristics of the different criteria are illustrated using bootstrap samples from the Nile data set. [ABSTRACT FROM AUTHOR]

R offers several extension packages that allow it to perform parallel computations. These operate on fixed points in the program flow and make it difficult to deal with nested parallelism and to organize parallelism in complex computations in general. In this article we discuss, first, of how to detect parallelism in functions, and second, how to minimize user intervention in that process. We present a solution that requires minimal code changes and enables to flexibly and dynamically choose the degree of parallelization in the resulting computation. An implementation is provided by the R package parallelize.dynamic and practical issues are discussed with the help of examples. [ABSTRACT FROM AUTHOR]

We studied wet-milling kinetics of a drug in a stirred mill using a microhydrodynamic model with various radial distribution functions (RDFs). An n th-order kinetics model was fitted to the median size evolution data at several stirrer speeds and loadings of polystyrene/zirconia beads to identify the breakage rate constant k. Microhydrodynamic parameters were calculated using three RDFs: Carnahan–Starling, Lun, and Ma–Ahmadi. The first one, used in previous microhydrodynamic studies, does not account for the bead packing limit concentration. The Lun and the Ma–Ahmadi RDFs similarly predicted much higher frequency of less energetic–forceful bead–bead collisions than the Starling RDF. A subset selection algorithm determined the best multiple linear regression models (MLRMs) of k with the microhydrodynamic parameters (semi-theoretical) or the process–bead parameters (empirical) as predictors. The Lun RDF-based MLRM had the best fitting–predictions of the kinetics among the semi-theoretical MLRMs, which outperformed the empirical MLRMs. [Display omitted] • Bead packing impact on the breakage kinetics and microhydrodynamics (MHD) in WSMM • Carnahan–Starling, Lun, and Ma–Ahmadi radial distribution functions (RDFs) compared • MLRMs correlated breakage rate constant to MHD parameters and process parameters • Best MLRMs by subset selection algorithm with a leave-one-out cross-validation • The MLRM with the Lun RDF-based MHD parameters has the best prediction capability [ABSTRACT FROM AUTHOR]