Search

Large Language Model (LLM) leaderboards based on benchmark rankings are regularly used to guide practitioners in model selection. Often, the published leaderboard rankings are taken at face value - we show this is a (potentially costly) mistake. Under existing leaderboards, the relative performance of LLMs is highly sensitive to (often minute) details. We show that for popular multiple choice question benchmarks (e.g. MMLU) minor perturbations to the benchmark, such as changing the order of choices or the method of answer selection, result in changes in rankings up to 8 positions. We explain this phenomenon by conducting systematic experiments over three broad categories of benchmark perturbations and identifying the sources of this behavior. Our analysis results in several best-practice recommendations, including the advantage of a hybrid scoring method for answer selection. Our study highlights the dangers of relying on simple benchmark evaluations and charts the path for more robust evaluation schemes on the existing benchmarks.

Healthcare Informatics is a phenomenon being talked about from the early 21st century in the era in which we are living. With evolution of new computing technologies huge amount of data in healthcare is produced opening several research areas. Managing the massiveness of this data is required while extracting knowledge for decision making is the main concern of today. For this task researchers are doing explorations in big data analytics, deep learning (advanced form of machine learning known as deep neural nets), predictive analytics and various other algorithms to bring innovation in healthcare. Through all these innovations happening it is not wrong to establish that disease prediction with anticipation of its cure is no longer unrealistic. First, Dengue Fever (DF) and then Covid-19 likewise are new outbreak in infectious lethal diseases and diagnosing at all stages is crucial to decrease mortality rate. In case of Diabetes, clinicians and experts are finding challenging the timely diagnosis and analyzing the chances of developing underlying diseases. In this paper, Louvain Mani-Hierarchical Fold Learning healthcare analytics, a hybrid deep learning technique is proposed for medical diagnostics and is tested and validated using real-time dataset of 104 instances of patients with dengue fever made available by Holy Family Hospital, Pakistan and 810 instances found for infectious diseases including prognosis of; Covid-19, SARS, ARDS, Pneumocystis, Streptococcus, Chlamydophila, Klebsiella, Legionella, Lipoid, etc. on GitHub. Louvain Mani-Hierarchical Fold Learning healthcare analytics showed maximum 0.952 correlations between two clusters with Spearman when applied on 240 instances extracted from comorbidities diagnostic data model derived from 15696 endocrine records of multiple visits of 100 patients identified by a unique ID. Accuracy for induced rules is evaluated by Laplace (Fig. 8) as 0.727, 0.701 and 0.203 for 41, 18 and 24 rules, respectively. Endocrine diag

Postpartum haemorrhage (PPH) is the loss of blood above 500 ml during vaginal or caesarean deliveries. It is difficult to find a PPH in an earlier stage, so pregnant women are exposed to excess blood loss that makes them suffer and die. Antenatal practices help in identifying risk factors, and modern technology is used to overcome the risk. Still, the morbidity rate and the mortality arise due to the unpredicted and unexpected cause. PPH is still a significant cause of maternal morbidity and mortality worldwide. The novelty of this research work is to alert medical practitioner about excessive bleeding of pregnant women during childbirth. We are proposing an automation system using wearable devices to prevent pregnant women from the PPH. These devices measure parameters like temperature, pulse rate, blood pressure, and sweat rate of pregnant women. Fuzzy neural technique-based rules are used for each parameter to predict the risk in developing PPH and to evaluate the performance of proposed system for reducing mortality and morbidity rates. Our findings of experiment are carried on metrics of HPPH (high-level postpartum haemorrhage), NPPH (normal-level postpartum haemorrhage), and MPPH (medium-level postpartum haemorrhage) for 15 patients. Fuzzy output value of 1 indicates patient state with NPPH, 0 indicates patient state with HPPH, and values in between 0 and 1 indicate MPPH. Based on the sensitivity of the predicted values, medical attention is taken from doctors or nurses in nearby locations using Internet of Things infrastructure.

Manually reading and logging gauge data is time inefficient, and the effort increases according to the number of gauges available. We present a computer vision pipeline that automates the reading of analog gauges. We propose a two-stage CNN pipeline that identifies the key structural components of an analog gauge and outputs an angular reading. To facilitate the training of our approach, a synthetic dataset is generated thus obtaining a set of realistic analog gauges with their corresponding annotation. To validate our proposal, an additional real-world dataset was collected with 4.813 manually curated images. When compared against state-of-the-art methodologies, our method shows a significant improvement of 4.55 in the average error, which is a 52% relative improvement. The resources for this project will be made available at: https://github.com/fuankarion/automatic-gauge-reading.

The global demand for electricity has visualized high growth with the rapid growth in population and economy. It thus becomes necessary to efficiently distribute electricity to households and industries in order to reduce power loss. Smart Grids (SG) have the potential to reduce such power losses during power distribution. Machine learning and artificial intelligence techniques have been successfully implemented on SGs to achieve enhanced accuracy in customer demand prediction. There exists a dire need to analyze and evaluate the various machine learning algorithms, thereby identify the most suitable one to be applied to SGs. In the present work, several state-of-the-art machine learning algorithms, namely Support Vector Machines (SVM), K-Nearest Neighbor (KNN), Logistic Regression, Naive Bayes, Neural Networks, and Decision Tree classifier, have been deployed for predicting the stability of the SG. The SG dataset used in the study is publicly available collected from UC Irvine (UCI) machine learning repository. The experimentation results highlighted the superiority of the Decision Tree classification algorithm, which outperformed the other state of the art algorithms yielding 100% precision, 99.9% recall, 100% F1 score, and 99.96% accuracy.

Postpartum haemorrhage (PPH) is the loss of blood above 500 ml during vaginal or caesarean deliveries. It is difficult to find a PPH in an earlier stage, so pregnant women are exposed to excess blood loss that makes them suffer and die. Antenatal practices help in identifying risk factors, and modern technology is used to overcome the risk. Still, the morbidity rate and the mortality arise due to the unpredicted and unexpected cause. PPH is still a significant cause of maternal morbidity and mortality worldwide. The novelty of this research work is to alert medical practitioner about excessive bleeding of pregnant women during childbirth. We are proposing an automation system using wearable devices to prevent pregnant women from the PPH. These devices measure parameters like temperature, pulse rate, blood pressure, and sweat rate of pregnant women. Fuzzy neural technique-based rules are used for each parameter to predict the risk in developing PPH and to evaluate the performance of proposed system for reducing mortality and morbidity rates. Our findings of experiment are carried on metrics of HPPH (high-level postpartum haemorrhage), NPPH (normal-level postpartum haemorrhage), and MPPH (medium-level postpartum haemorrhage) for 15 patients. Fuzzy output value of 1 indicates patient state with NPPH, 0 indicates patient state with HPPH, and values in between 0 and 1 indicate MPPH. Based on the sensitivity of the predicted values, medical attention is taken from doctors or nurses in nearby locations using Internet of Things infrastructure.

The COVID-19 pandemic has caused hundreds of thousands of deaths, millions of infections worldwide, and the loss of trillions of dollars for many large economies. It poses a grave threat to the human population with an excessive number of patients constituting an unprecedented challenge with which health systems have to cope. Researchers from many domains have devised diverse approaches for the timely diagnosis of COVID-19 to facilitate medical responses. In the same vein, a wide variety of research studies have investigated underlying medical conditions for indicators suggesting the severity and mortality of, and role of age groups and gender on, the probability of COVID-19 infection. This study aimed to review, analyze, and critically appraise published works that report on various factors to explain their relationship with COVID-19. Such studies span a wide range, including descriptive analyses, ratio analyses, cohort, prospective and retrospective studies. Various studies that describe indicators to determine the probability of infection among the general population, as well as the risk factors associated with severe illness and mortality, are critically analyzed and these findings are discussed in detail. A comprehensive analysis was conducted on research studies that investigated the perceived differences in vulnerability of different age groups and genders to severe outcomes of COVID-19. Studies incorporating important demographic, health, and socioeconomic characteristics are highlighted to emphasize their importance. Predominantly, the lack of an appropriated dataset that contains demographic, personal health, and socioeconomic information implicates the efficacy and efficiency of the discussed methods. Results are overstated on the part of both exclusion of quarantined and patients with mild symptoms and inclusion of the data from hospitals where the majority of the cases are potentially ill.

The global demand for electricity has visualized high growth with the rapid growth in population and economy. It thus becomes necessary to efficiently distribute electricity to households and industries in order to reduce power loss. Smart Grids (SG) have the potential to reduce such power losses during power distribution. Machine learning and artificial intelligence techniques have been successfully implemented on SGs to achieve enhanced accuracy in customer demand prediction. There exists a dire need to analyze and evaluate the various machine learning algorithms, thereby identify the most suitable one to be applied to SGs. In the present work, several state-of-the-art machine learning algorithms, namely Support Vector Machines (SVM), K-Nearest Neighbor (KNN), Logistic Regression, Naive Bayes, Neural Networks, and Decision Tree classifier, have been deployed for predicting the stability of the SG. The SG dataset used in the study is publicly available collected from UC Irvine (UCI) machine learning repository. The experimentation results highlighted the superiority of the Decision Tree classification algorithm, which outperformed the other state of the art algorithms yielding 100% precision, 99.9% recall, 100% F1 score, and 99.96% accuracy.

The COVID-19 pandemic has caused hundreds of thousands of deaths, millions of infections worldwide, and the loss of trillions of dollars for many large economies. It poses a grave threat to the human population with an excessive number of patients constituting an unprecedented challenge with which health systems have to cope. Researchers from many domains have devised diverse approaches for the timely diagnosis of COVID-19 to facilitate medical responses. In the same vein, a wide variety of research studies have investigated underlying medical conditions for indicators suggesting the severity and mortality of, and role of age groups and gender on, the probability of COVID-19 infection. This study aimed to review, analyze, and critically appraise published works that report on various factors to explain their relationship with COVID-19. Such studies span a wide range, including descriptive analyses, ratio analyses, cohort, prospective and retrospective studies. Various studies that describe indicators to determine the probability of infection among the general population, as well as the risk factors associated with severe illness and mortality, are critically analyzed and these findings are discussed in detail. A comprehensive analysis was conducted on research studies that investigated the perceived differences in vulnerability of different age groups and genders to severe outcomes of COVID-19. Studies incorporating important demographic, health, and socioeconomic characteristics are highlighted to emphasize their importance. Predominantly, the lack of an appropriated dataset that contains demographic, personal health, and socioeconomic information implicates the efficacy and efficiency of the discussed methods. Results are overstated on the part of both exclusion of quarantined and patients with mild symptoms and inclusion of the data from hospitals where the majority of the cases are potentially ill.

In this paper, we present an identity (ID) based protocol that secures AODV and TCP so that it can be used in dynamic and attack prone environments of mobile ad hoc networks. The proposed protocol protects AODV using Sequential Aggregate Signatures (SAS) based on RSA. It also generates a session key for each pair of source-destination nodes of a MANET for securing the end-to-end transmitted data. Here each node has an ID which is evaluated from its public key and the messages that are sent are authenticated with a signature/ MAC. The proposed scheme does not allow a node to change its ID throughout the network lifetime. Thus it makes the network secure against attacks that target AODV and TCP in MANET. We present performance analysis to validate our claim.