Search

In the past 40 years, therapeutic antibody discovery and development have advanced considerably, with machine learning (ML) offering a promising way to speed up the process by reducing costs and the number of experiments required. Recent progress in ML-guided antibody design and development (D&D) has been hindered by the diversity of data sets and evaluation methods, which makes it difficult to conduct comparisons and assess utility. Establishing standards and guidelines will be crucial for the wider adoption of ML and the advancement of the field. This perspective critically reviews current practices, highlights common pitfalls and proposes method development and evaluation guidelines for various ML-based techniques in therapeutic antibody D&D. Addressing challenges across the ML process, best practices are recommended for each stage to enhance reproducibility and progress., (Copyright © 2024 The Author(s). Published by Elsevier Ltd.. All rights reserved.)

In many modern machine learning applications, changes in covariate distributions and difficulty in acquiring outcome information have posed challenges to robust model training and evaluation. Numerous transfer learning methods have been developed to robustly adapt the model itself to some unlabeled target populations using existing labeled data in a source population. However, there is a paucity of literature on transferring performance metrics, especially receiver operating characteristic (ROC) parameters, of a trained model. In this paper, we aim to evaluate the performance of a trained binary classifier on unlabeled target population based on ROC analysis. We proposed Semisupervised Transfer lEarning of Accuracy Measures (STEAM), an efficient three-step estimation procedure that employs (1) double-index modeling to construct calibrated density ratio weights and (2) robust imputation to leverage the large amount of unlabeled data to improve estimation efficiency. We establish the consistency and asymptotic normality of the proposed estimator under the correct specification of either the density ratio model or the outcome model. We also correct for potential overfitting bias in the estimators in finite samples with cross-validation. We compare our proposed estimators to existing methods and show reductions in bias and gains in efficiency through simulations. We illustrate the practical utility of the proposed method on evaluating prediction performance of a phenotyping model for rheumatoid arthritis (RA) on a temporally evolving EHR cohort., (© The Author(s) 2024. Published by Oxford University Press on behalf of The International Biometric Society.)

Background: The reporting of machine learning (ML) prognostic and diagnostic modeling studies is often inadequate, making it difficult to understand and replicate such studies. To address this issue, multiple consensus and expert reporting guidelines for ML studies have been published. However, these guidelines cover different parts of the analytics lifecycle, and individually, none of them provide a complete set of reporting requirements., Objective: We aimed to consolidate the ML reporting guidelines and checklists in the literature to provide reporting items for prognostic and diagnostic ML in in-silico and shadow mode studies., Methods: We conducted a literature search that identified 192 unique peer-reviewed English articles that provide guidance and checklists for reporting ML studies. The articles were screened by their title and abstract against a set of 9 inclusion and exclusion criteria. Articles that were filtered through had their quality evaluated by 2 raters using a 9-point checklist constructed from guideline development good practices. The average κ was 0.71 across all quality criteria. The resulting 17 high-quality source papers were defined as having a quality score equal to or higher than the median. The reporting items in these 17 articles were consolidated and screened against a set of 6 inclusion and exclusion criteria. The resulting reporting items were sent to an external group of 11 ML experts for review and updated accordingly. The updated checklist was used to assess the reporting in 6 recent modeling papers in JMIR AI. Feedback from the external review and initial validation efforts was used to improve the reporting items., Results: In total, 37 reporting items were identified and grouped into 5 categories based on the stage of the ML project: defining the study details, defining and collecting the data, modeling methodology, model evaluation, and explainability. None of the 17 source articles covered all the reporting items. The study details and data description reporting items were the most common in the source literature, with explainability and methodology guidance (ie, data preparation and model training) having the least coverage. For instance, a median of 75% of the data description reporting items appeared in each of the 17 high-quality source guidelines, but only a median of 33% of the data explainability reporting items appeared. The highest-quality source articles tended to have more items on reporting study details. Other categories of reporting items were not related to the source article quality. We converted the reporting items into a checklist to support more complete reporting., Conclusions: Our findings supported the need for a set of consolidated reporting items, given that existing high-quality guidelines and checklists do not individually provide complete coverage. The consolidated set of reporting items is expected to improve the quality and reproducibility of ML modeling studies., (©William Klement, Khaled El Emam. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 31.08.2023.)

Overfitting describes the phenomenon where a highly predictive model on the training data generalizes poorly to future observations. It is a common concern when applying machine learning techniques to contemporary medical applications, such as predicting vaccination response and disease status in infectious disease or cancer studies. This review examines the causes of overfitting and offers strategies to counteract it, focusing on model complexity reduction, reliable model evaluation, and harnessing data diversity. Through discussion of the underlying mathematical models and illustrative examples using both synthetic data and published real datasets, our objective is to equip analysts and bioinformaticians with the knowledge and tools necessary to detect and mitigate overfitting in their research.

Background: Assessing the performance of machine learning (ML) models requires careful consideration of the evaluation metrics used. It is often necessary to utilize multiple metrics to gain a comprehensive understanding of a trained model's performance, as each metric focuses on a specific aspect. However, comparing the scores of these individual metrics for each model to determine the best-performing model can be time-consuming and susceptible to subjective user preferences, potentially introducing bias., Results: We propose the Machine Learning Cumulative Performance Score (MLcps), a novel evaluation metric for classification problems. MLcps integrates several precomputed evaluation metrics into a unified score, enabling a comprehensive assessment of the trained model's strengths and weaknesses. We tested MLcps on 4 publicly available datasets, and the results demonstrate that MLcps provides a holistic evaluation of the model's robustness, ensuring a thorough understanding of its overall performance., Conclusions: By utilizing MLcps, researchers and practitioners no longer need to individually examine and compare multiple metrics to identify the best-performing models. Instead, they can rely on a single MLcps value to assess the overall performance of their ML models. This streamlined evaluation process saves valuable time and effort, enhancing the efficiency of model evaluation. MLcps is available as a Python package at https://pypi.org/project/MLcps/., (© The Author(s) 2023. Published by Oxford University Press GigaScience.)

Background: Machine learning algorithms are currently used in a wide array of clinical domains to produce models that can predict clinical risk events. Most models are developed and evaluated with retrospective data, very few are evaluated in a clinical workflow, and even fewer report performances in different hospitals. In this study, we provide detailed evaluations of clinical risk prediction models in live clinical workflows for three different use cases in three different hospitals., Objective: The main objective of this study was to evaluate clinical risk prediction models in live clinical workflows and compare their performance in these setting with their performance when using retrospective data. We also aimed at generalizing the results by applying our investigation to three different use cases in three different hospitals., Methods: We trained clinical risk prediction models for three use cases (ie, delirium, sepsis, and acute kidney injury) in three different hospitals with retrospective data. We used machine learning and, specifically, deep learning to train models that were based on the Transformer model. The models were trained using a calibration tool that is common for all hospitals and use cases. The models had a common design but were calibrated using each hospital's specific data. The models were deployed in these three hospitals and used in daily clinical practice. The predictions made by these models were logged and correlated with the diagnosis at discharge. We compared their performance with evaluations on retrospective data and conducted cross-hospital evaluations., Results: The performance of the prediction models with data from live clinical workflows was similar to the performance with retrospective data. The average value of the area under the receiver operating characteristic curve (AUROC) decreased slightly by 0.6 percentage points (from 94.8% to 94.2% at discharge). The cross-hospital evaluations exhibited severely reduced performance: the average AUROC decreased by 8 percentage points (from 94.2% to 86.3% at discharge), which indicates the importance of model calibration with data from the deployment hospital., Conclusions: Calibrating the prediction model with data from different deployment hospitals led to good performance in live settings. The performance degradation in the cross-hospital evaluation identified limitations in developing a generic model for different hospitals. Designing a generic process for model development to generate specialized prediction models for each hospital guarantees model performance in different hospitals., (©Hong Sun, Kristof Depraetere, Laurent Meesseman, Patricia Cabanillas Silva, Ralph Szymanowsky, Janis Fliegenschmidt, Nikolai Hulde, Vera von Dossow, Martijn Vanbiervliet, Jos De Baerdemaeker, Diana M Roccaro-Waldmeyer, Jörg Stieg, Manuel Domínguez Hidalgo, Fried-Michael Dahlweid. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 07.06.2022.)

Machine learning is becoming an integral part of the Design-Build-Test-Learn cycle in biotechnology. Machine learning models learn from collected datasets such as omics data and predict a defined outcome, which has led to both production improvements and predictive tools in the field. Robust prediction of the behavior of microbial cell factories and production processes not only greatly increases our understanding of the function of such systems, but also provides significant savings of development time. However, many pitfalls when modeling biological data - bad fit, noisy data, model instability, low data quantity and imbalances in the data - cause models to suffer in their performance. Here we provide an accessible, in-depth analysis on the problems created by these pitfalls, as well as means of their detection and mediation, with a focus on supervised learning. Assessing the state of the art, we show that, currently, in-depth analyses of model performance are often absent and must be improved. This review provides a toolbox for the analysis of model robustness and performance, and simultaneously proposes a standard for the community to facilitate future work. It is further accompanied by an interactive online tutorial on the discussed issues., (Copyright © 2021 Elsevier Inc. All rights reserved.)

Complex disease is different from Mendelian disorders. Its development usually involves the interaction of multiple genes or the interaction between genes and the environment (i.e. epistasis). Although the high-throughput sequencing technologies for complex diseases have produced a large amount of data, it is extremely difficult to analyze the data due to the high feature dimension and the combination in the epistasis analysis. In this work, we introduce machine learning methods to effectively reduce the gene dimensionality, retain the key epistatic effects, and effectively characterize the relationship between epistatic effects and complex diseases.

Model selection and performance assessment for prediction models are important tasks in machine learning, e.g. for the development of medical diagnosis or prognosis rules based on complex data. A common approach is to select the best model via cross-validation and to evaluate this final model on an independent dataset. In this work, we propose to instead evaluate several models simultaneously. These may result from varied hyperparameters or completely different learning algorithms. Our main goal is to increase the probability to correctly identify a model that performs sufficiently well. In this case, adjusting for multiplicity is necessary in the evaluation stage to avoid an inflation of the family wise error rate. We apply the so-called maxT-approach which is based on the joint distribution of test statistics and suitable to (approximately) control the family-wise error rate for a wide variety of performance measures. We conclude that evaluating only a single final model is suboptimal. Instead, several promising models should be evaluated simultaneously, e.g. all models within one standard error of the best validation model. This strategy has proven to increase the probability to correctly identify a good model as well as the final model performance in extensive simulation studies.

Background: Thrombin is the central protease of the vertebrate blood coagulation cascade, which is closely related to cardiovascular diseases. The inhibitory constant Ki is the most significant property of thrombin inhibitors., Method: This study was carried out to predict Ki values of thrombin inhibitors based on a large data set by using machine learning methods. Taking advantage of finding non-intuitive regularities on high-dimensional datasets, machine learning can be used to build effective predictive models. A total of 6554 descriptors for each compound were collected and an efficient descriptor selection method was chosen to find the appropriate descriptors. Four different methods including multiple linear regression (MLR), K Nearest Neighbors (KNN), Gradient Boosting Regression Tree (GBRT) and Support Vector Machine (SVM) were implemented to build prediction models with these selected descriptors., Results: The SVM model was the best one among these methods with R2=0.84, MSE=0.55 for the training set and R2=0.83, MSE=0.56 for the test set. Several validation methods such as yrandomization test and applicability domain evaluation, were adopted to assess the robustness and generalization ability of the model. The final model shows excellent stability and predictive ability and can be employed for rapid estimation of the inhibitory constant, which is full of help for designing novel thrombin inhibitors., (Copyright© Bentham Science Publishers; For any queries, please email at epub@benthamscience.net.)

Nowadays, high latency in Edge Computing (EC) for Internet of Things (IoT) devices due to network congestion and online traffic reduces the acquired precision, performance, and processing power of the network. Data overload in IoT significantly impacts the real-time capabilities of user experience, decision- making efficiency, operational costs, and security in EC. By combining EC innovation and three Machine Learning (ML) models, namely Decision Trees (DT), Support Vector Machines (SVMs), and Convolutional Neural Networks (CNNs), this research aims to tackle the inactivity of IoT devices and information cleaning from errors. Its purpose is to preserve information astuteness and highlight the efficacy of each model's execution by using the essential components of previous approaches. The proposed model evaluates the precision, performance, and quality enhancement by measuring the Mean Square Error (MSE), coefficient of determination (R²), and accuracy. [ABSTRACT FROM AUTHOR]

Today, geomechanics plays a crucial role in the oil industry, particularly in enhancing production and ensuring well stability. To achieve optimal results, accurate estimation of geomechanical parameters is essential. One of the low-cost and accurate methods for estimating geomechanical parameters is the use of intelligent methods. In this research, geomechanical parameters are estimated using conventional data logs using intelligent methods. The aim of this study is to introduce a new machine learning algorithm to estimate geomechanical parameters using conventional data logs in one of the hydrocarbon field wells in southwest Iran. In this article, the shear wave velocity and uniaxial compressive strength (UCS) were estimated using machine learning algorithms. Subsequently, other geomechanical parameters were calculated based on these estimated parameters derived from machine learning algorithms. For shear wave velocity (Vs) prediction using MLP and CLM (CNN+LSTM+MLP) algorithms, First, effective features were selected using the Auto-encoder deep learning algorithm. The selected features for Vs input into the algorithms were Vp, RHOB, CALIPER, and NPHI, and then the Vs is estimated with MLP and CLM algorithm. To evaluate the results, the model was assessed using metrics such as MAE, MAPE, MSE, RMSE, NRMSE, and R2 on the train, test, and blind datasets. The CLM algorithm consistently demonstrated superior performance across all datasets, including training, testing, and blind data sets. The R2 values for blind data were $R_{MLP}^2 = 0.8727,$ R M L P 2 = 0.8727 , $R_{CLM}^2 = 0.9274$ R C L M 2 = 0.9274 , respectively. These outputs are crucial for estimating subsequent studies. Next, Elastic Young's moduli and Poisson's ratio were calculated, and the dynamic brittleness index was computed using dynamic Young's modulus and Poisson's ratio. Subsequently, UCS values were predicted using machine learning algorithms. Since there were 12 laboratory core samples of UCS available, UCS was initially calculated using relevant empirical relations and data from some available well logs. This process aimed to extrapolate these core results to cover the entire target depth range from 3551.072 to 3799.789 m. Subsequently, a relationship between UCS derived from well logs and laboratory results was established for the depths corresponding to where the laboratory samples were recorded. Following that, an autoencoder deep network was utilized to select effective features for predicting UCS. The selected features for UCS input into the algorithms were Vp, RHOB, and CALIPER. Subsequently, UCS was estimated using MLP and CLM algorithms. To evaluate the results, the model performance was assessed using metrics such as MAE, MAPE, MSE, RMSE, NRMSE, and R2 on the train, test, and blind datasets. Furthermore, the results were checked and evaluated using a set of statistical measures calculated for the train, test, and blind datasets, that $ R_{MLP}^2 = 0.9305, R_{CLM}^2 = 0.9953 $ R M L P 2 = 0.9305 , R C L M 2 = 0.9953 were obtained for blind UCS data. The results demonstrate that CLM achieves high accuracy in estimating these parameters, with deep learning algorithms showing higher determination coefficients and lower errors compared to MLP. Furthermore, UCS and tensile strength were calculated, followed by the computation of static brittleness index. The relationship between dynamic and static brittleness index was also investigated. Overall, the findings indicate that machine learning algorithms are robust and accurate methods for estimating Vs, UCS, and other geomechanical parameters using conventional logs. [ABSTRACT FROM AUTHOR]

There has been an explosion of data collected about sports. Because such data is extremely rich and complex, machine learning is increasingly being used to extract actionable insights from it. Typically, machine learning is used to build models and indicators that capture the skills, capabilities, and tendencies of athletes and teams. Such indicators and models are in turn used to inform decision-making at professional clubs. Designing these indicators requires paying careful attention to a number of subtle issues from a methodological and evaluation perspective. In this paper, we highlight these challenges in sports and discuss a variety of approaches for handling them. Methodologically, we highlight that dependencies affect how to perform data partitioning for evaluation as well as the need to consider contextual factors. From an evaluation perspective, we draw a distinction between evaluating the developed indicators themselves versus the underlying models that power them. We argue that both aspects must be considered, but that they require different approaches. We hope that this article helps bridge the gap between traditional sports expertise and modern data analytics by providing a structured framework with practical examples. [ABSTRACT FROM AUTHOR]

Using skewed sequential data, the study explores the effectiveness of numerous sequential models designed for binary classification tasks. The dataset under investigation consists of 5,595 testing samples and 13,055 training samples, a structure that presents significant difficulties because of uneven labelling. The researchers carefully go through pretreatment procedures, which include text data encoding and effective methods for handling missing information, in order to address this. The study employs and examines a wide range of algorithms, which reflects the heterogeneous sequential modelling environment. A variety of neural network architectures are included in the arsenal: CNN, CNN-RNN, RCNN. The binary classification job at hand is used to thoroughly assess each architecture, revealing both its advantages and disadvantages. The study's evaluation approach, which presents a wide range of measures indicating consistently excellent performance overall, is its key component. Among these algorithms stand out as the best with an astounding 97% accuracy rate on a variety of evaluation metrics. This strong performance highlights their ability to handle sequential data with unbalanced labels and establishes a standard for further work in related fields. Beyond its empirical results, the study is important because it provides a well-designed assessment approach that may be used as a benchmark by practitioners facing similar problems. Through the clarification of important concepts related to model selection and performance evaluation, the study provides professionals and academics with crucial resources to efficiently traverse the complex terrain of sequential modelling. [ABSTRACT FROM AUTHOR]

Purpose: Particle size distribution (PSD) assessment, which affects all physical, chemical, biological, mineralogical, and geological properties of soil, is crucial for maintaining soil sustainability. It plays a vital role in ensuring appropriate land use, fertilizer management, crop selection, and conservation practices, especially in fragile soils such as those of the North-Western Himalayas. Materials and methods: In this study, the performance of eleven mathematical and three Machine Learning (ML) models used in the past was compared to investigate PSD modeling of different soils from the North-Western Himalayan region, considering that an appropriate model must fit all PSD data. Results and discussion: Our study focuses on the significance of evaluating the goodness of fit in particle size distribution modeling using the coefficient of determination (R2adj = 0.79 to 0.45), the Akaike information criterion (AIC = 67 to 184), and the root mean square error (RMSE = 0.01 to 0.09). The Fredlund, Weibull, and Rosin Rammler models exhibited the best fit for all samples, while the performance of the Gompertz, S-Curve, and Van Genutchen models was poor. Of the three ML models tested, the Random Forest model performed the best (R2 = 0.99), and the SVM model was the lowest (R2 = 0.95). Thus, the PSD of the soil can be best predicted by ML approaches, especially by the Random Forest model. Conclusion: The Fredlund model exhibited the best fit among mathematical models while random forest performed best among the machine learning models. As the number of parameters in the model increased better was the accuracy. [ABSTRACT FROM AUTHOR]

Breast cancer is ranked as a significant cause of mortality among females globally. Its complex nature poses principal challenges for physicians and researchers for rapid diagnosis and prognosis. Hence, machine learning algorithms are employed to forecast and identify diseases. This study discusses the comparative analysis of seven machine learning models, e.g., logistic regression (LR), support vector machine (SVM), k-nearest neighbor classifier (KNN), decision tree classifier (DT), random forest classifier (RF), Naïve Bayes (NB), and artificial neural network (ANN) to predict breast cancer using Wisconsin breast cancer and breast cancer datasets. In the Wisconsin breast cancer dataset, KNN depicted 99% accuracy, followed by RF (98%), SVM (96%), NB (96%), LR (96%), ANN (93%), and DT (92%). On the contrary, in the breast cancer (BC) dataset, the highest accuracy was achieved by LR at 83%, and the lowest was achieved by DT (65%), which depicted that the numeric dataset WBC has better accuracy than the breast cancer dataset. [ABSTRACT FROM AUTHOR]

This study leverages sophisticated machine learning methodologies, particularly XGBoost, to analyze cardiovascular diseases through cardiac datasets. The methodology encompasses meticulous data pre-processing, training of the XGBoost algorithm, and its performance evaluation using metrics such as accuracy, precision, and ROC curves. This technique represents a notable progression in the realm of medical research, potentially leading to enhanced diagnostic precision and a deeper comprehension of cardiovascular ailments, thereby improving patient care and treatment modalities in cardiology. Furthermore, the research delves into the utilization of deep learning methodologies for the automated delineation of cardiac structures in MRI and mammography images, aiming to boost diagnostic precision and patient management. [24][3][5][6] In assessing machine learning algorithms' efficacy in diagnosing cardiovascular diseases, this analysis underscores the pivotal role of such algorithms and their possible data inputs. Additionally, it investigates promising directions for future exploration, such as the application of reinforcement learning. A significant aspect of our investigation is the development and deployment of sophisticated deep learning models for segmenting right ventricular images from cardiac MRI scans, aiming at heightened accuracy and dependability in diagnostics. Through the utilization of advanced techniques like Fourier Convolutional Neural Network (FCNN) and improved versions of Vanilla Convolutional Neural Networks (Vanilla-CNN) and Residual Networks (ResNet), we achieved a substantial improvement in accuracy and reliability. This enhancement allows for more precise and quicker identification and diagnosis of cardiovascular diseases, which is of utmost importance in clinical practice. Evaluation of Machine Learning Algorithms: We conducted a comprehensive evaluation of machine learning algorithms in the context of cardiovascular disease diagnosis. This assessment emphasized the fundamental role of machine learning algorithms and their potential data sources. We also explored promising avenues, such as reinforcement learning, for future research. Factors Affecting Predictive Models: We highlighted the critical factors affecting the effectiveness of machine learning-based predictive models. These factors include data heterogeneity, depth, and breadth, as well as the nature of the modeling task, and the choice of algorithms and feature selection methods. Recognizing and addressing these factors are essential for building reliable models. [ABSTRACT FROM AUTHOR]

Interactions among atmospheric, root‐soil, and vegetation processes drive carbon dioxide fluxes (Fc) from land to atmosphere. Eddy covariance measurements are commonly used to measure Fc at sub‐daily timescales and validate process‐based and data‐driven models. However, these validations do not reveal process interactions, thresholds, and key differences in how models replicate them. We use information theory‐based measures to explore multivariate information flow pathways from forcing data to observed and modeled hourly Fc, using flux tower data sets in the Midwestern U.S. in intensively managed corn‐soybean landscapes. We compare multiple linear regressions, long‐short term memory, and random forests (RF), and evaluate how different model structures use information from combinations of sources to predict Fc. We extend a framework for model predictive and functional performance, which examines a suite of dependencies from all forcing variables to the observed or modeled target. Of the three model types, RF exhibited the highest functional and predictive performance, correctly capturing strong dependencies between radiation and temperature variables with Fc. Regionally trained models demonstrate lower predictive but higher functional performance compared to site‐specific models, suggesting superior reproduction of observed relationships at the expense of predictive accuracy. This study shows that some metrics of predictive performance encapsulate functional behaviors better than others, highlighting the need for multiple metrics of both types. This study improves our understanding of carbon fluxes in an intensively managed landscape, and more generally provides insight into how model structures and forcing variables translate to interactions that are well versus poorly captured in models. Plain Language Summary: In an agricultural landscape, exchanges of carbon dioxide between the land and atmosphere occur due to photosynthesis and respiration and depend on weather, soil, and vegetation conditions. Traditional model performance metrics focus on the relationship between observed and modeled outputs, while functional performance considers the relationships between interacting inputs and outputs. We compare several performance measures for three different machine learning (ML) models that simulate sub‐daily carbon fluxes. We look at how drivers such as solar radiation, soil moisture, temperature, humidity, and rainfall provide information to carbon fluxes, and whether different ML models also capture these interactions. In other words: Air, soil, and plants drive carbon's upward path,Models are detectives, interpreting their math.With information theory, we map data's travel courses,To see how models find or miss carbon's causal sources. Key Points: Information theory measures describe individual and joint causal relationships in observed versus modeled vertical carbon dioxide fluxesThree machine learning models overestimate unique information from sources at the expense of synergistic, or joint informationRegionally trained models have improved functional but not predictive performances, indicating a trade‐off [ABSTRACT FROM AUTHOR]

Business failure prediction has become crucially important for today's firms, enabling them to reduce financial risks and make informed decisions. This study uses a dataset of 6819 companies and 96 financial and macroeconomic variables to present a comparative analysis of machine learning (ML) models for predicting corporate bankruptcies. Behind this research is to improve the accuracy of bankruptcy prediction, which can help companies make more informed decisions and reduce financial risks. This study aims to evaluate the effectiveness of 16 ML algorithms in terms of accuracy, sensitivity, and other relevant metrics. The work uses methodologies that include data collection and cleaning, exploratory data analysis, model preprocessing and training, and model performance evaluation. Data preprocessing and hyperparameter optimization techniques were used to improve model performance. The evaluated algorithms include Classifiers such as Stacking Classifier (SCC), Randomized Search Classifier (RCV), Historical Gradient Boosting Classifier (HGBC), MLP Classifier (MLPC), K-Neighbors Classifier (KNC), Decision Tree Classifier (DTC), XGBRF Classifier (XGBRFC), Support Vector Classifier (SVC), Logistic Regression Classifier (LR), Linear SVC Classifier (LSVC). With an accuracy of 97.63 %, recall of 97.63 %, and F1-score of 97.63 %, the results show that the SCC algorithm was the best. Other models, such as RCV and DTC, also showed good results, with accuracies above 97 %. However, models such as PAC and BNB had lower performance and accuracy below 90 %. Finally, this study compares the results of ML models in predicting business failures and highlights their effectiveness. The SCC algorithm is considered the most suitable model for this task, as it suggests that it can help economic actors make more informed decisions and reduce financial risks in the context of firms.

In machine learning or scientific computing, model performance is measured with an objective function. But why choose one objective over another? According to the information‐theoretic paradigm, the "best" objective function is whichever minimizes information loss. To evaluate different objectives, transform them into likelihoods. The ratios of these likelihoods represent how strongly we should prefer one objective versus another, and the log of that ratio represents the relative information loss (or gain) from one objective to another. In plain terms, minimizing information loss is equivalent to minimizing uncertainty, as well as maximizing probability and general utility. We argue that this paradigm is well‐suited to models that have many uses and no definite utility like the complex Earth system models used to understand the effects of climate change. Furthermore, the benefits of "maximizing information and general utility" extend beyond model accuracy to other important considerations including how efficiently the model calibrates, how well it generalizes, and how well it compresses data. Key Points: A basic problem in modeling is the choice of objective function (or performance metric)According to information theory, the "best" objective function minimizes information loss, which we evaluate using the AICLike friction or inefficiency in a system, information loss incurs additional cost however the model is used [ABSTRACT FROM AUTHOR]

Machine learning is increasingly and ubiquitously being used in the medical domain. Evaluation metrics like accuracy, precision, and recall may indicate the performance of the models but not necessarily the reliability of their outcomes. This paper assesses the effectiveness of a number of machine learning algorithms applied to an important dataset in the medical domain, specifically, mental health, by employing explainability methodologies. Using multiple machine learning algorithms and model explainability techniques, this work provides insights into the models' workings to help determine the reliability of the machine learning algorithm predictions. The results are not intuitive. It was found that the models were focusing significantly on less relevant features and, at times, unsound ranking of the features to make the predictions. This paper therefore argues that it is important for research in applied machine learning to provide insights into the explainability of models in addition to other performance metrics like accuracy. This is particularly important for applications in critical domains such as healthcare. [ABSTRACT FROM AUTHOR]

Non-coding variants associated with complex traits can alter the motifs of transcription factor (TF)–deoxyribonucleic acid binding. Although many computational models have been developed to predict the effects of non-coding variants on TF binding, their predictive power lacks systematic evaluation. Here we have evaluated 14 different models built on position weight matrices (PWMs), support vector machines, ordinary least squares and deep neural networks (DNNs), using large-scale in vitro (i.e. SNP-SELEX) and in vivo (i.e. allele-specific binding, ASB) TF binding data. Our results show that the accuracy of each model in predicting SNP effects in vitro significantly exceeds that achieved in vivo. For in vitro variant impact prediction, kmer/gkm-based machine learning methods (deltaSVM_HT-SELEX, QBiC-Pred) trained on in vitro datasets exhibit the best performance. For in vivo ASB variant prediction, DNN-based multitask models (DeepSEA, Sei, Enformer) trained on the ChIP-seq dataset exhibit relatively superior performance. Among the PWM-based methods, tRap demonstrates better performance in both in vitro and in vivo evaluations. In addition, we find that TF classes such as basic leucine zipper factors could be predicted more accurately, whereas those such as C2H2 zinc finger factors are predicted less accurately, aligning with the evolutionary conservation of these TF classes. We also underscore the significance of non-sequence factors such as cis -regulatory element type, TF expression, interactions and post-translational modifications in influencing the in vivo predictive performance of TFs. Our research provides valuable insights into selecting prioritization methods for non-coding variants and further optimizing such models. [ABSTRACT FROM AUTHOR]

Maintenance and safety inspection of trains is a critical element of providing a safe and reliable train service. Checking for the presence of bolts is an essential part of train inspection, which is currently, typically carried out during visual inspections. There is an opportunity to automate bolt inspection using machine vision with edge devices. One particular challenge is the implementation of such inspection mechanisms on edge devices, which necessitates using lighter models to ensure efficiency. Traditional methods have often fallen short of the required object detection performance, thus demonstrating the need for a more advanced approach. To address this challenge, researchers have been exploring the use of deep learning algorithms and computer vision techniques to improve the accuracy and reliability of bolt detection on edge devices. High precision in identifying absent bolts in train components is essential to avoid potential mishaps and system malfunctions. This paper presents "BoltVision", a comparative analysis of three cutting-edge machine learning models: convolutional neural networks (CNNs), vision transformers (ViTs), and compact convolutional transformers (CCTs). This study illustrates the superior assessment capabilities of these models and discusses their effectiveness in addressing the prevalent issue of edge devices. Results show that BoltVision, utilising a pre-trained ViT base, achieves a remarkable 93% accuracy in classifying missing bolts. These results underscore the potential of BoltVision in tackling specific safety inspection challenges for trains and highlight its effectiveness when deployed on edge devices characterised by constrained computational resources. This attests to the pivotal role of transformer-based architectures in revolutionising predictive maintenance and safety assurance within the rail transportation industry. [ABSTRACT FROM AUTHOR]

Three-River-Source (TRS) National Park stands as one of China's earliest established national parks, dedicated to significant ecological responsibilities that include conserving soil and water resources in the Tibetan Plateau region. Research on climate change's influence on the TRS region's grasslands is of great significance in our efforts to comprehend and conserve the grassland ecosystem. The most effective random forest (RF) model was chosen to invert the aboveground biomass (AGB) of grassland in the previous 6 years (2015-2020) and predict the grassland AGB in the following 20 years (2021-2040) by comparing linear regression and multivariate nonlinear regression models such as RF, support vector machine, decision tree, and artificial neural network. A Theil-Sen median trend analysis and a Mann-Kendal test were then used to examine the trends of grassland AGB. The results showed that (1) RF outperformed other models in estimating grassland AGB, with a test set decision coefficient of multiple determination (R²) of 0.722, a root mean square error of 42.596 g/m², and a mean absolute error of 35.619 g/m²; (2) over 6 years, the grassland AGB in TRS National Park had a spatial trend of a steady rise from the northwest to the southeast. The average annual grassland AGB was 247.333 g/m², with averages of 44.836 g/m² , 92.601 g/m² , and 120.217 g/m² in the Yangtze River, Yellow River, and Lancang River source parks respectively. The trend of the grassland AGB was primarily stabilized and slightly recovered, with a small portion of the slightly deteriorated areas; (3) climate change significantly affected grassland AGB, and when temperature and precipitation conditions were adequate, grassland AGB values increased with temperature and precipitation. In the scenarios of ssp119, ssp245, and ssp585, grassland AGB is projected to exhibit a dynamic upward trend over the next 20 years. Global warming is expected to boost grassland AGB. Comprehensive measures are essential to maintain grassland health and ensure a positive impact on global carbon and ecological balance. The study's findings hold great importance for the ecological security of the TRS region and contribute to our global understanding of sustainable grassland development. [ABSTRACT FROM AUTHOR]

The escalating adoption of Machine Learning techniques has given a bigger picture to newbies trying to explore the usage of it. Our tool deals with the idea of helping them, in order to make their lives easier. Various visualizations and algorithms have been developed to help Machine Learning enthusiasts decide upon the best model. Deciding the perfect model needs enough time which now can be reduced by the solution provided in this tool. This interactive technique helps users with some expertise to explore and validate predictive as well as classification models. Once the user provides the dataset, the visualization techniques discussed in this tool lets the user decide to select the features that are most suitable for training the model. It allows one to decide upon the importance of a particular feature and know the dataset predictions across various algorithms used for regression and classification. The accuracy percentage or the precision, recall score for regression and classification models respectively can be seen in order to know the best model. [ABSTRACT FROM AUTHOR]

During the process of food storage and circulation, there will be different degrees of quality deterioration. With the improvement of people's attention to food quality and safety, it is of great significance to carry out quality prediction research in the process of food storage and transportation for quality control. This paper reviews the research progress of machine learning in food storage quality prediction, including conventional quality prediction methods and limitations, and then focuses on the rapid development and wide application of integrated learning and artificial neural network algorithms, and prediction performance evaluation methods in recent years. Finally, it summarizes and looks forward to the future development trend of machine learning in the food field, and provides relevant references for the development of food science cross research. [ABSTRACT FROM AUTHOR]

Weather forecasting has always been challenging due to the atmosphere's complex and dynamic nature. Weather conditions such as rain, clouds, clear skies, and sunniness are influenced by several factors, including temperature, pressure, humidity, wind speed, and direction. Physical and complex models are currently used to determine weather conditions, but they have their limitations, particularly in terms of computing time. In recent years, supervised machine learning methods have shown great potential in predicting weather events accurately. These methods use historical weather data to train a model, which can then be used to predict future weather conditions. This study enhances weather forecasting by employing four supervised machine learning techniques—artificial neural networks (ANNs), support vector machines (SVMs), random forest (RF), and k-nearest neighbors (KNN)—on three distinct datasets obtained from the Weatherstack database. These datasets, with varying temporal spans and uncertainty levels in their input features, are used to train and evaluate the methods. The results show that the ANN has superior performance across all datasets. Furthermore, when compared to Weatherstack's weather prediction model, all methods demonstrate significant improvements. Interestingly, our models show variance in performance across different datasets, particularly those with predicted rather than observed input features, underscoring the complexities of handling data uncertainty. The study provides valuable insights into the use of supervised machine learning techniques for weather forecasting and contributes to the development of more precise prediction models. [ABSTRACT FROM AUTHOR]

In Morocco, agriculture is an important sector that contributes to the country's economy and food security. Accurately predicting crop yields is crucial for farmers, policy makers, and other stakeholders to make informed decisions regarding resource allocation and food security. This paper investigates the potential of Machine Learning algorithms for improving the accuracy of crop yield predictions in Morocco. The study examines various factors that affect crop yields, including weather patterns, soil moisture levels, and rainfall, and how these factors can be incorporated into Machine Learning models. The performance of different algorithms, including Decision Trees, Random Forests, and Neural Networks, is evaluated and compared to traditional statistical models used for crop prediction. The study demonstrated that the Machine Learning algorithms outperformed the Statistical models in predicting crop yields. Specifically, the Machine Learning algorithms achieved mean squared error values between 0.10 and 0.23 and coefficient of determination values ranging from 0.78 to 0.90, while the Statistical models had mean squared error values ranging from 0.16 to 0.24 and coefficient of determination values ranging from 0.76 to 0.84. The Feed Forward Artificial Neural Network algorithm had the lowest mean squared error value (0.10) and the highest R² value (0.90), indicating that it performed the best among the three Machine Learning algorithms. These results suggest that Machine Learning algorithms can significantly improve the accuracy of crop yield predictions in Morocco, potentially leading to improved food security and optimized resource allocation for farmers. [ABSTRACT FROM AUTHOR]

In this paper, a set of best practice data sharing guidelines for wind turbine fault detection model evaluation is developed, which can help practitioners overcome the main challenges of digitalisation. Digitalisation is one of the key drivers for reducing costs and risks over the whole wind energy project life cycle. One of the largest challenges in successfully implementing digitalisation is the lack of data sharing and collaboration between organisations in the sector. In order to overcome this challenge, a new collaboration framework called WeDoWind was developed in recent work. The main innovation of this framework is the way it creates tangible incentives to motivate and empower different types of people from all over the world to share data and knowledge in practice. In this present paper, the challenges related to comparing and evaluating different SCADA-data-based wind turbine fault detection models are investigated by carrying out a new case study, the "WinJi Gearbox Fault Detection Challenge", based on the WeDoWind framework. A total of six new solutions were submitted to the challenge, and a comparison and evaluation of the results show that, in general, some of the approaches (Particle Swarm Optimisation algorithm for constructing health indicators, performance monitoring using Deep Neural Networks, Combined Ward Hierarchical Clustering and Novelty Detection with Local Outlier Factor and Time-to-failure prediction using Random Forest Regression) appear to exhibit high potential to reach the goals of the Challenge. However, there are a number of concrete things that would have to have been done by the Challenge providers and the Challenge moderators in order to ensure success. This includes enabling access to more details of the different failure types, access to multiple data sets from more wind turbines experiencing gearbox failure, provision of a model or rule relating fault detection times or a remaining useful lifetime to the estimated costs for repairs, replacements and inspections, provision of a clear strategy for training and test periods in advance, as well as provision of a pre-defined template or requirements for the results. These learning outcomes are used directly to define a set of best practice data sharing guidelines for wind turbine fault detection model evaluation. The guidelines can be used by researchers in the sector in order to improve model evaluation and data sharing in the future. [ABSTRACT FROM AUTHOR]

The diagnosis and prognosis of cardiovascular disease are crucial medical tasks to ensure correct classification, which helps cardiologists provide proper treatment to the patient. Machine learning applications in the medical niche have increased as they can recognize patterns from data. Using machine learning to classify cardiovascular disease occurrence can help diagnosticians reduce misdiagnosis. This research develops a model that can correctly predict cardiovascular diseases to reduce the fatality caused by cardiovascular diseases. This paper proposes a method of k-modes clustering with Huang starting that can improve classification accuracy. Models such as random forest (RF), decision tree classifier (DT), multilayer perceptron (MP), and XGBoost (XGB) are used. GridSearchCV was used to hypertune the parameters of the applied model to optimize the result. The proposed model is applied to a real-world dataset of 70,000 instances from Kaggle. Models were trained on data that were split in 80:20 and achieved accuracy as follows: decision tree: 86.37% (with cross-validation) and 86.53% (without cross-validation), XGBoost: 86.87% (with cross-validation) and 87.02% (without cross-validation), random forest: 87.05% (with cross-validation) and 86.92% (without cross-validation), multilayer perceptron: 87.28% (with cross-validation) and 86.94% (without cross-validation). The proposed models have AUC (area under the curve) values: decision tree: 0.94, XGBoost: 0.95, random forest: 0.95, multilayer perceptron: 0.95. The conclusion drawn from this underlying research is that multilayer perceptron with cross-validation has outperformed all other algorithms in terms of accuracy. It achieved the highest accuracy of 87.28%. [ABSTRACT FROM AUTHOR]

Background Assessing the performance of machine learning (ML) models requires careful consideration of the evaluation metrics used. It is often necessary to utilize multiple metrics to gain a comprehensive understanding of a trained model's performance, as each metric focuses on a specific aspect. However, comparing the scores of these individual metrics for each model to determine the best-performing model can be time-consuming and susceptible to subjective user preferences, potentially introducing bias. Results We propose the Machine Learning Cumulative Performance Score (MLcps), a novel evaluation metric for classification problems. MLcps integrates several precomputed evaluation metrics into a unified score, enabling a comprehensive assessment of the trained model's strengths and weaknesses. We tested MLcps on 4 publicly available datasets, and the results demonstrate that MLcps provides a holistic evaluation of the model's robustness, ensuring a thorough understanding of its overall performance. Conclusions By utilizing MLcps, researchers and practitioners no longer need to individually examine and compare multiple metrics to identify the best-performing models. Instead, they can rely on a single MLcps value to assess the overall performance of their ML models. This streamlined evaluation process saves valuable time and effort, enhancing the efficiency of model evaluation. MLcps is available as a Python package at https://pypi.org/project/MLcps/. [ABSTRACT FROM AUTHOR]

Recently, self-supervised pretraining of transformers has gained considerable attention in analyzing electronic medical records. However, systematic evaluation of different pretraining tasks in radiology applications using both images and radiology reports is still lacking. We propose PreRadE, a simple proof of concept framework that enables novel evaluation of pretraining tasks in a controlled environment. We investigated three most-commonly used pretraining tasks (MLM—Masked Language Modelling, MFR—Masked Feature Regression, and ITM—Image to Text Matching) and their combinations against downstream radiology classification on MIMIC-CXR, a medical chest X-ray imaging and radiology text report dataset. Our experiments in the multimodal setting show that (1) pretraining with MLM yields the greatest benefit to classification performance, largely due to the task-relevant information learned from the radiology reports. (2) Pretraining with only a single task can introduce variation in classification performance across different fine-tuning episodes, suggesting that composite task objectives incorporating both image and text modalities are better suited to generating reliably performant models. [ABSTRACT FROM AUTHOR]

Business failure prediction has become crucially important for today's firms, enabling them to reduce financial risks and make informed decisions. This study uses a dataset of 6819 companies and 96 financial and macroeconomic variables to present a comparative analysis of machine learning (ML) models for predicting corporate bankruptcies. Behind this research is to improve the accuracy of bankruptcy prediction, which can help companies make more informed decisions and reduce financial risks. This study aims to evaluate the effectiveness of 16 ML algorithms in terms of accuracy, sensitivity, and other relevant metrics. The work uses methodologies that include data collection and cleaning, exploratory data analysis, model preprocessing and training, and model performance evaluation. Data preprocessing and hyperparameter optimization techniques were used to improve model performance. The evaluated algorithms include Classifiers such as Stacking Classifier (SCC), Randomized Search Classifier (RCV), Historical Gradient Boosting Classifier (HGBC), MLP Classifier (MLPC), K-Neighbors Classifier (KNC), Decision Tree Classifier (DTC), XGBRF Classifier (XGBRFC), Support Vector Classifier (SVC), Logistic Regression Classifier (LR), Linear SVC Classifier (LSVC). With an accuracy of 97.63 %, recall of 97.63 %, and F1-score of 97.63 %, the results show that the SCC algorithm was the best. Other models, such as RCV and DTC, also showed good results, with accuracies above 97 %. However, models such as PAC and BNB had lower performance and accuracy below 90 %. Finally, this study compares the results of ML models in predicting business failures and highlights their effectiveness. The SCC algorithm is considered the most suitable model for this task, as it suggests that it can help economic actors make more informed decisions and reduce financial risks in the context of firms. [ABSTRACT FROM AUTHOR]

[Display omitted] An inherent difference exists between male and female bodies, the historical under-representation of females in clinical trials widened this gap in existing healthcare data. The fairness of clinical decision-support tools is at risk when developed based on biased data. This paper aims to quantitatively assess the gender bias in risk prediction models. We aim to generalize our findings by performing this investigation on multiple use cases at different hospitals. First, we conduct a thorough analysis of the source data to find gender-based disparities. Secondly, we assess the model performance on different gender groups at different hospitals and on different use cases. Performance evaluation is quantified using the area under the receiver-operating characteristic curve (AUROC). Lastly, we investigate the clinical implications of these biases by analyzing the underdiagnosis and overdiagnosis rate, and the decision curve analysis (DCA). We also investigate the influence of model calibration on mitigating gender-related disparities in decision-making processes. Our data analysis reveals notable variations in incidence rates, AUROC, and over-diagnosis rates across different genders, hospitals and clinical use cases. However, it is also observed the underdiagnosis rate is consistently higher in the female population. In general, the female population exhibits lower incidence rates and the models perform worse when applied to this group. Furthermore, the decision curve analysis demonstrates there is no statistically significant difference between the model's clinical utility across gender groups within the interested range of thresholds. The presence of gender bias within risk prediction models varies across different clinical use cases and healthcare institutions. Although inherent difference is observed between male and female populations at the data source level, this variance does not affect the parity of clinical utility. In conclusion, the evaluations conducted in this study highlight the significance of continuous monitoring of gender-based disparities in various perspectives for clinical risk prediction models. [ABSTRACT FROM AUTHOR]

We contribute a novel model evaluation technique that divides available measurements into training and testing sets in a way that adheres to the requirements imposed on professional monitoring stations. We perform extensive and systematic experiments with a wide range of state-of-the-art calibration models to demonstrate that our approach provides accurate insights about the performance of calibration models in realworld deployments, while at the same time highlighting issues with evaluation techniques used in previous works. Among others, our results show that although trained and tested in the same location, calibration errors can exhibit deviation up to 116% depending on the evaluation protocol that is being adopted. We also demonstrate that models trained with continuous data can suffer up to 76% greater error when tested with data coming from diverse environmental conditions. In contrast, when models are trained and tested with our method, the variability of errors is significantly reduced and the robustness of calibration models is significantly improved. The overall performance improvements depend on pollutant concentration, ranging from 10% for low concentrations to 90% for high concentrations that represent conditions that are most dangerous for human health. [ABSTRACT FROM AUTHOR]

In the field of computer research, the increase of data in result of societal progress has been remarkable, and the management of this data and the analysis of linked businesses have grown in popularity. There are numerous practical uses for the capability to extract key characteristics from secondary property data and utilize these characteristics to forecast home prices. Using regression methods in machine learning to segment the data set, examine the major factors affecting it, and forecast home prices is the most popular method for examining pricing information. It is challenging to generate precise forecasts since many of the regression models currently being utilized in research are unable to efficiently collect data on the distinctive elements that correlate y with a high degree of house price movement. In today's forecasting studies, ensemble learning is a very prevalent and well-liked study methodology. The regression integration computation of large housing datasets can use a lot of computer resources as well as computation time, and ensemble learning uses more resources and calls for more machine support in integrating diverse models. The Average Model suggested in this paper uses the concept of fusion to produce integrated analysis findings from several models, combining the best benefits of separate models. The Average Model has a strong applicability in the field of regression prediction and significantly increases computational efficiency. The technique is also easier to replicate and very effective in regression investigations. Before using regression processing techniques, this work creates an average of different regression models using theAM(Average Model) algorithm in a novelway. By evaluating essential models with 90% accuracy, this technique significantly increases the accuracy of house price predictions. The experimental results show that the AM algorithm proposed in this paper has lower prediction error than other comparison algorithms, and the prediction accuracy is greatly improved compared with other algorithms, and has a good experimental effect in house price prediction. [ABSTRACT FROM AUTHOR]