Search

The use of machine learning in building technology has become increasingly important in recent years. One of the applications is heating load prediction, which enables demand-side flexibility. Most studies consider the heating load prediction without sufficient context with the existing building characteristics. For an accurate load prediction, suitable features have to be selected according to their importance, the feature importance (FI). The scope of this paper is to investigate whether there is a relationship between the building characteristics and the FI and if so, how strong this relationship is. Additionally, an analysis has been conducted to determine which building characteristic have the most significant impact on FI. For this purpose, a full factorial design of a room with six different building characteristics is carried out. In total, the heating load is calculated for 15 552 room variants. The thermal balance, correlation, random forest FI, permutation FI and SHapley Additive exPlanations (SHAP) values are calculated for these different rooms. The local SHAP values were used to explain the model. These values also provide insight into the interaction of individual features with the heating load. For most variants, the outdoor temperature had the highest FI. It is investigated which building characteristics have the greatest influence on the thermal balance, correlation, FI and SHAP values. A relationship was found between the proportion of thermal balance, the correlation between the features and the label as well as the FI. The greatest association with the thermal balance characteristics was found for the SHAP values. The study shows the systematic relationship between building characteristics and FI. Therefore, FI should always be considered in the context of building characteristics. [Display omitted] • SHAP values provide an excellent way of showing how features affect heating load. • Comparison of various feature selection methods for 15 552 different rooms. • Characteristics of the building are reflected in the feature importance. • Air exchange rate has the greatest influence on the feature importance. [ABSTRACT FROM AUTHOR]

Predicting the mode of child birth is still remains one of the most complex and challenging tasks in ancient times. Also, there is no such strong methodologies are developed in the conventional works for birth mode prediction. Therefore, the proposed work objects to develop a novel and distinct optimization based machine learning technique for creating the child birth mode prediction system. This framework includes the modules of data imputation, feature selection, classification, and prediction. Initially, the data imputation process is performed to improve the quality of dataset by normalizing the attributes and filling the missed fields. Then, the Multivariate Intensified Mine Blast Optimization (MIMBO) technique is implemented to choose the best set of features by estimating the optimal function. After that, an integrated Naïve Bayes – Random Forest (NBRF) technique is developed by incorporating the functions of conventional NB and RF techniques. The novel contribution of this technique, a Bird Mating (BM) optimization technique is used in NBRF classifier for estimating the likelihood parameter to generate the Bayesian rules. The main idea of this paper is to develop a simple as well as efficient automated system with the use of hybrid machine learning model for predicting the mode of child birth. For this purpose, advanced algorithms such as MIMBO based feature selection, and NBRF based classification are implemented in this work. Due to the inclusion of MIMBO and BM optimization techniques, the performance of classifier is greatly improved with low computational burden and increased prediction accuracy. Moreover, the combination of proposed MIMBO-NBRF technique outperforms the existing child birth prediction methods with superior results in terms of average accuracy up to 99 %. In addition, some other parameters are also estimated and compared with the existing techniques for proving the overall superiority of the proposed framework. • The data imputation process is performed to improve the quality of dataset. • Multivariate Intensified Mine Blast Optimization (MIMBO) technique is implemented. • An integrated Naïve Bayes – Random Forest (NBRF) technique is developed. • Bird Mating (BM) optimization technique is used in NBRF classifier to estimate the likelihood parameter. [ABSTRACT FROM AUTHOR]

• The mutual information method was used as a feature selection method for choosing geological and hydraulic fracturing features. • The Gaussian process regression (GPR) method was applied to predict the open hole flow rate after fracturing work. • The performances of the GPR, random forest, and ANN models were compared based on the actual engineering data of 175 sample wells. • The Shapley additive explanation value method was applied to obtain the dominant factors influencing the initial gas production rate. In shale gas production rate prediction, data are commonly acquired through simulations, which offer benefits but cannot fully predict the outcome of real engineering projects. This paper applied a Gaussian process regression (GPR) model, which was trained on the geological and field data from 175 wells in the Fuling gas field, China. GPR was chosen because this method can be suitably applied to a small dataset. To improve the accuracy, reduce the computational consumption, and increase the interpretability of the model, the mutual information method was used for feature selection after Pearson coefficient analysis. Based on comparison of the GPR, random forest and artificial neural network models, GPR achieved the best performance with the mutual information feature selection method, exhibiting R2 values of 0.71 and 0.68 for the training and test datasets, respectively. After model training, the model applied the Shapley additive explanation (SHAP) value method. The results indicate that geological factors dominate the gas production rate in Fuling, China. The volume of fluid loss could exert a negative impact on production rates and increase the consumption of 40–70 mesh proppant could increase gas production in Fuling. [ABSTRACT FROM AUTHOR]

Age estimation has become inordinately significant for human beings for many reasons, such as detecting legal and criminal responsibility and other social events like a marriage license, birth certificate, etc. This paper aims to decide on the most desirable machine learning algorithm (from conventional machine learning algorithms to deep learning) for dental age estimation based on buccal bone level. The database consisted of 150 CBCT images (73 males and 77 females) from an existing base of the Faculty of Dental Medicine with Clinics, University of Sarajevo, aged 20-69. Results were obtained using the Waikato Environment for Knowledge Analysis (Weka), machine learning software in Java. Left and Right Buccal Alveolar Bone Levels are increasing with age, so they showed to be the most important attributes, especially the latter. Random Forest classifier provided the greatest result with the correlation coefficient of 0.803 and the mean absolute error of 6.022. We have also shown that considering sinus-related features can be a significant addition to the databases. Our paper is probably one of the first studies where regression algorithms based on the Support Vector Machines and Random Forest were utilized. [ABSTRACT FROM AUTHOR]

Paroxysmal atrial fibrillation (PAF) is a cardiac arrhythmia which is often challenging to diagnose because patients may be asymptomatic, and episodes are usually intermittent. In this paper we describe a method for identifying patients with a history of PAF using electrocardiogram (ECG) recordings during sinus rhythm. We analyzed, on a beat-to-beat basis, the P-waves in the sinus rhythm ECGs of 69 patients with a history of PAF and 59 healthy individuals. From each subject's P-waves, we calculated key electrocardiographic metrics, including some which are proposed here for the first time. Using means testing and feature selection methods, we discerned the features which were most useful for classification, and trained a Random Forest which identified PAF patients. Our approach achieved a 93.45% accuracy, sensitivity of 95.21%, and specificity of 91.40% using P-wave integral and novel amplitude and slope-based features which ranked highest in importance compared to other metrics from the literature. In particular, descriptive statistics of P-wave amplitudes, slopes, and integrals, were effective for identifying subjects with PAF history vs. healthy individuals. Our method has a high sensitivity and discrimination ability, with an AUC=0.9669 which is superior to others', and can thus be potentially valuable for the early identification of patients who are prone to episodes of PAF, even as part of the standard cardiological checkup that most adults undergo periodically. • Early identification of PAF during sinus rhythm with high accuracy. • P-wave signal metrics on a beat-to-beat basis are proposed. • P-wave amplitude, integral and slope features were ranked as the most important. • A RF classifier distinguishes PAF from healthy subjects with a state-of-the-art AUC. [ABSTRACT FROM AUTHOR]

• Forecast electricity demand considering residents' psychological preferences. • Stationarity time series are established to avoid the impact of irregular factors. • Personalizes exogenous variables, combines correlations and importance. • A hybrid machine learning based model is built to improve forecasting performance. • Multiple implications are provided for the government and institutions. Faced with the explosive growth of population and rapid development of industrialization, household electricity demand (HED) has recently become an essential sector of energy demand. Very-short-term forecasting of HED is closely related to monitoring insecurity, detecting emergency and potential crisis. Consider challenges triggered by the characteristics of time series, firstly, a phase of data pre-processing with stationarity of time series is constructed to manage irregular trend factors. Subsequently, in the feature selection phase, a two-layer feature selection method is proposed with the combination of random forest and autocorrelation analysis to select the most relevant features, which simultaneously considers correlations and importance of input lagged variables as well as exogenous variables. After that, hybrid HED forecasting methods are constructed with incorporating multiple machine learning techniques in the forecasting phase, namely k-nearest neighbor, support vector regression , random forest and multilayer perceptron, using their respective advantages of managing different situations. As residents' psychological preferences and calendar variables highly affect future HED, this paper intends to add and test their effects on HED. Finally, the proposed methods are examined on a 10-min-ahead horizon application, whose forecasting results, comparison analyses and an extended experiment are given to illustrate their effectiveness, after which some implications are provided to show the insights gained from the results. [ABSTRACT FROM AUTHOR]

• A data-driven product failure detection framework was developed. • Sampling strategies improved the performance of the models. • ANOVA selection technique improved the performance. • The RUSBoosted Tree algorithm provided satisfactory results. • The proposed methodology is flexible to be applied in other sectors. For a healthy production line, it is essential to ensure a low failure rate of products. Product quality in production lines can be inspected using several techniques at the end of a production process, including a manual inspection. Different methods are applied to inspect the product quality at the end of the production process and sometimes during the production. This is often done using manual inspection, but this is less efficient, expensive, and time-consuming. Machine learning algorithms have the potential for evaluating and predicting product quality in a production line. In this paper, a novel product failure detection model that applies ANOVA (Analysis of Variance) feature selection method, Min-Max Scaling normalization method, mean imputation technique, Random Forest classification algorithm, a data sampling technique, and Grid Search parameter optimization approach is proposed and validated. For the comparison of the proposed model, several experiments have been performed using five classification algorithms, including RUSBoosted Tree. Experimental results demonstrated that the proposed model using the Random Forest algorithm, ANOVA feature selection, and sampling method achieves the best performance among other models and detects the faulty products effectively. It was also shown that the RUSBoosted Tree algorithm can be considered by practitioners for building the faulty product prediction model when data sampling and feature selection techniques are not integrated into the prediction model. [ABSTRACT FROM AUTHOR]

Today, a significant number of biomedical data is generated continuously from various biomedical equipment and experiments due to rapid technological improvements in medical sciences. Effective biomedical data analysis, such as extracting biological and diagnostically significant features, is a very challenging task. This paper proposes hybrid Machine Learning Classification Techniques based on ensemble technique with Enhanced-Grey Wolf Optimization (E-GWO) feature selection algorithm to analyze these complex biomedical data. We combined five biomedical heart disease data sets for the experimental work: Cleveland, Long-Beach-VA, Switzerland, Hungarians, and Statlog. New hybrid Machine Learning Classification Techniques classifiers like Naive Bayes Bagging Technique (NBBT), Random Forest Bagging Technique (RFBT), Decision Tree Bagging Technique (DTBT), K-Nearest Neighbors Bagging Technique (KNNBT), Neural Network Bagging Technique (NNBT), Gradient Boosting Boosting Technique (GBBT), and Adaptive Boosting Boosting Technique (ABBT) are developed with bagging and boosting methods. Accuracy, Recall, Precision, F1-score, Specificity, Error Rate, G-mean, False Negative Rate (FNR), False Positive Rate (FPR), and Negative Predictive Value (NPV) are used to evaluate hybrid techniques. The experimental result shows that the developed hybrid classifier RFBT achieves the highest accuracy of 99.26% with E-GWO feature selection. The proposed method has 11.90% improved the accuracy to the conventional model. [ABSTRACT FROM AUTHOR]

Corporate financial distress prediction research has been ongoing for more than half a century, during which many models have emerged, among which ensemble learning algorithms are the most accurate. Most of the state-of-the-art methods of recent years are based on gradient boosted decision trees. However, most of them do not consider using feature importance for feature selection, and a few of them use the feature importance method with bias, which may not reflect the true importance of features. To solve this problem, a heuristic algorithm based on permutation importance (PIMP) is proposed to modify the biased feature importance measure in this paper. This method ranks and filters the features used by machine learning models, which not only improves accuracy but also makes the results more interpretable. Based on financial data from 4,167 listed companies in China between 2001 and 2019, the experiment shows that compared with using the random forest (RF) wrapper method alone, the bias in feature importance is indeed corrected by combining the PIMP method. After the redundant features are removed, the performance of most machine learning models is improved. The PIMP method is a promising addition to the existing financial distress prediction methods. Moreover, compared with traditional statistical learning models and other machine learning models, the proposed PIMP-XGBoost offers higher prediction accuracy and clearer interpretation, making it suitable for commercial use. • The model Combines a corrected feature selection measure and XGBoost. • Permutation importance can correct the bias of feature importance. • The model is validated on Chinese listed companies datasets over five metrics. • The model is proved to outperform several benchmark techniques. • The feature importance and partial dependence plot enhance model interpretation. [ABSTRACT FROM AUTHOR]

Machine Learning techniques (ML) are being widely adopted in embedded devices due to their efficiency and flexibility. However, the strict power limitations in such devices, combined with the variable resource requirements of ML models, require further understanding of how model complexity affects power and performance. This paper proposes a framework that facilitates the design space exploration of dedicated decision trees (DT) and random forests (RF) accelerators by enabling a joint assessment of power dissipation and prediction accuracy. The proposed framework translates tree-based structures to hardware description languages (HDL). The HDL modules are submitted to logic and physically-aware hardware synthesis flows, allowing a detailed power-performance analysis of VLSI DTs and RFs. Using four data sets of embedded applications as case studies, we found that quantizing the input features leads to accuracy gains of up to 6.3% compared with the precise versions. We also show that using shallower trees may lead to small prediction loss with significant reductions in power, which is favorable for power-constrained applications. Our translator achieves better results in terms of energy/inference w.r.t. prior related works under comparison, one of which employed standard methods for hardware translation such as High-Level Synthesis. The proposed solution presents a power reduction of 10 times or more for the same inference throughput reported in prior works. [ABSTRACT FROM AUTHOR]

• FCS is proposed as a complementary step for principal component analysis. • PCA-FCS identifies the most relevant components based on their level of contribution. • PCA-FCS simultaneously performs dimension reduction and component selection. • PCA-FCS has substantially improved the accuracy of the random forest algorithm. Principal component analysis (PCA) is one the most effective and widely used dimensionality-reduction techniques which aggregates the maximal variance in the first few components. In many applications, the first few components of PCA are used in place of the original variables in statistical and machine learning models; nevertheless, such use does not guarantee the selection of the most relevant components to the target variable. This paper presents an efficient approach in which all components, rather than simply the first few, are considered as input for the random forest (RF) model, and a feature selection algorithm is integrated with the RF to select the most relevant components, hence called fully component selection (FCS). The proposed method was evaluated on spectroscopic data comprising 70 soil samples with 2050 spectral bands to estimate soil organic carbon (SOC). The FCS approach was compared to the RF model once considering the original features, then again using only the first few components, and again using all components without feature selection. The results showed that RF-FCS substantially outperformed other approaches, such that the R2 was increased between 25.2% and 55.5%. Furthermore, the findings indicated that the most relevant principal components (PCs) to the target variable were not the first few, but PC6, PC7, PC8, PC15, and PC43. Using the FCS approach increased model performance substantively, and coupling it with machine learning and statistical models is strongly recommended for high dimensional data applications. [ABSTRACT FROM AUTHOR]

• A machine learning network traffic analysis system has been designed. • Experiments of our proposed machine learning classifiers on a real-world P2P network traffic classification problem. • P2P network traffic classification using various machine learning techniques. • Comparison Performance of all Ensemble and Single models was performed using VPN netoworks. Network traffic classification is a significant and problematic aspect of network resource management arising from an investigation of network developments, planning, and design for 5G and beyond. Recently, traffic investigation systems for network monitoring and user access restrictions to Virtual Private Networks (VPN) and non-Virtual Private Networks (non-VPN) have gained widespread attention. In this paper, different algorithms for classifying and detecting VPN traffic are considered. A few existing machine learning procedures were tested concerning their performance in network traffic classification and security. The purpose is to improve Precision, Recall, and F1-score in VPN Network Traffic using Ensemble Classifiers. Therefore, the parameters of the ensemble classifier were changed to obtain high Precision, Recall, and F1-score. Bagging Decision Tree and Gradient Boosting algorithms were used for classification which produced promising results when compared to single classifiers like k -Nearest Neighbors (kNN), Multilayer Perceptron (MLP), and Decision Tree. The proposed classifier demonstrates recognition accuracy on a test sample of up to 93.80 % which outperforms all other single algorithms used in previous work. The MLP, Random Forest (RF), and Gradient Boosting (GB) algorithms had almost identical performance in all experiments. Furthermore, the proposed classifiers are found to perform better when the network traffic flows are generated using different values of time parameters (timeout). Our results show that the ensemble algorithms (Random Forest and the Gradient Boosting) outperform the single machine learning classifier previously used by other researchers, and we achieved the highest accuracy with the random forest classifier with better results while using non-VPN traffic and VPN traffic. The novelty lies in the application of an ensemble algorithm to secure a network traffic classification performed in comparison with single classifiers to determine Accuracy, Precision, and F1-score of a given dataset, contrary to the known process of selection of features and generation. [ABSTRACT FROM AUTHOR]