Search

This study introduces a novel systematic approach to address the challenge of labeled data scarcity for fault detection and diagnosis (FDD) in District Heating (DH) systems. To replicate real-world DH fault scenarios, we have created a controlled laboratory emulation of a generic DH substation integrated with a climate chamber. Furthermore, we present an FDD pipeline using an isolation forest and a one-class support vector machine for fault detection alongside a random forest and a support vector machine for fault diagnosis. Our research analyzed the impact of data sampling frequencies on the FDD models, revealing that shorter intervals, such as 1-min and 5-min, significantly improve FDD performance. We provide detailed information on six scenarios, including normal operation, a minor valve leak, a valve leak, a stuck valve, a high heat curve, and a temperature sensor deviation. For each scenario, we present their signature, quantifying their unique behavior and providing deeper insights into the operational implications. The signatures suggest that, while variable, faults have a consistent pattern seen in the generic DH substation. While this work contributes directly to the DH field, our methodology also extends its applicability to a broader context where labeled data is scarce. © 2024 The Authors

The success of Federated Learning (FL) hinges upon the active participation and contributions of edge devices as they collaboratively train a global model while preserving data privacy. Understanding the behavior of individual clients within the FL framework is essential for enhancing model performance, ensuring system reliability, and protecting data privacy. However, analyzing client behavior poses a significant challenge due to the decentralized nature of FL, the variety of participating devices, and the complex interplay between client models throughout the training process. This research proposes a novel approach based on eccentricity analysis to address the challenges associated with understanding the different clients' behavior in the federation. We study how the eccentricity analysis can be applied to monitor the clients' behaviors through the training process by assessing the eccentricity metrics of clients' local models and clients' data representation in the global model. The Kendall ranking method is used for evaluating the correlations between the defined eccentricity metrics and the clients' benefit from the federation and influence on the federation, respectively. Our initial experiments on a publicly available data set demonstrate that the defined eccentricity measures can provide valuable information for monitoring the clients' behavior and eventually identify clients with deviating behavioral patterns. © 2024 IEEE.

Federated learning (FL) enables edge nodes to collaboratively contribute to constructing a global model without sharing their data. This is accomplished by devices computing local, private model updates that are then aggregated by a server. However, computational resource constraints and network communication can become a severe bottleneck for larger model sizes typical for deep learning (DL) applications. Edge nodes tend to have limited hardware resources (RAM, CPU), and the network bandwidth and reliability at the edge is a concern for scaling federated fleet applications. In this paper, we propose and evaluate a FL strategy inspired by transfer learning in order to reduce resource utilization on devices, as well as the load on the server and network in each global training round. For each local model update, we randomly select layers to train, freezing the remaining part of the model. In doing so, we can reduce both server load and communication costs per round by excluding all untrained layer weights from being transferred to the server. The goal of this study is to empirically explore the potential trade-off between resource utilization on devices and global model convergence under the proposed strategy. We implement the approach using the FL framework FEDn. A number of experiments were carried out over different datasets (CIFAR-10, CASA, and IMDB), performing different tasks using different DL model architectures. Our results show that training the model partially can accelerate the training process, efficiently utilizes resources on-device, and reduce the data transmission by around 75% and 53% when we train 25%, and 50% of the model layers, respectively, without harming the resulting global model accuracy. Furthermore, our results demonstrate a negative correlation between the number of participating clients in the training process and the number of layers that need to be trained on each client’s side. As the number of clients increases, there is a decrease

Recently, there has been a substantial surge in interest surrounding diffusion models, which are considered a prominent class of generative models. This surge is primarily attributed to their potential applications in a variety of deep learning problems. The primary objective of this study is to assess the effectiveness of diffusion models as a data augmentation technique in the context of medical image analysis. Furthermore, it aims to conduct a comparative analysis of the performance exhibited by deep learning classifiers trained on two distinct datasets. One dataset is augmented using the diffusion model, while the other dataset undergoes traditional data augmentation techniques. Utilizing the IDRiD dataset for the purpose of diabetic retinopathy diagnosis, the results demonstrate the efficiency of the diffusion model as a data augmentation technique for medical images compared to traditional data augmentation techniques. The integration of diffusion model augmented data yields superior performance for both classifiers. Namely, the fine-tuned ResNet-50 reached an accuracy of 53.40%, and the proposed CNN-based approach reached an accuracy of 44.66%, surpassing the performance of classifiers trained using traditional data augmentation techniques.

Brain tumor segmentation, among various tasks in medical image analysis, has garnered significant attention in the research community. Despite continuous efforts by researchers, accurate brain tumor segmentation remains a key challenge. This challenge arises due to various factors, including location uncertainty, morphological uncertainty, low contrast imaging, annotation bias, and data imbalance. Magnetic resonance imaging (MRI) plays a vital role in providing detailed images of the brain, enabling the extraction of crucial information about the tumor’s shape, size, and location. In literature, deep learning algorithms have shown their efficiency in dealing with semantic segmentation, particularly the U-Net architecture. The latter has demonstrated impressive performance in Medical image segmentation. In this paper, a U-Net-based architecture for brain tumor segmentation is proposed. To further enhance the segmentation performance of our model, a novel ensemble learning method, EnsUNet, is introduced by integrating four pre-trained networks namely MobileNet, DeepLabV3+, ResNet, and DenseNet as the encoder within the U-Net architecture. The conducted experimental evaluation demonstrates promising results, achieving an Intersection over Union (IoU) score of 0.86, a Dice Coefficient (DC) of 0.92, and an accuracy of approximately 0.99. These findings underscore the effectiveness of our proposed EnsUNet for accurately segmenting brain tumors. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024.

Federated learning is a distributed paradigm that trained large-scale neural network models with the participation of multiple edge nodes and data remains on their devices, only sharing the local model updates. With this feature, federated learning is considered a secure solution for data privacy issues. However, the typical FL structure relies on the client–server architecture, which leads to the single-point-of-failure (SPoF) attack, and the random selection of edge devices for model training compromised the accuracy of the model. Furthermore, adversaries try to initiate inference attack i.e., attack on privacy leads to gradient leakage attack. Hence, we proposed a blockchain-based optimized edge node selection and privacy-preserved framework to address the aforementioned issues. We have designed three kinds of smart contracts (1) registration of edge nodes (2) forward bidding to select optimized edge devices for FL model training, and (3) payment settlement and reward smart contracts. Moreover, fully homomorphic encryption with the Cheon, Kim, Kim, and Song (CKKS) method is implemented before transmitting the local model updates to the server. Finally, we evaluated our proposed method on the benchmark dataset and compared it with other state-of-the-art studies. Consequently, we have achieved a higher accuracy and privacy-preserved FL framework with a decentralized nature. © 2023, The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.

Federated learning is a technique that allows to collaboratively train a shared machine learning model across distributed devices, where the data are stored locally on devices. Most innovations the research community proposes in federated learning are tested through custom simulators. An analysis of the literature shows the lack of workbench platforms for the performance evaluation of FL projects. This paper aims to fill the gap by presenting FLWB, a general-purpose, configurable, and scalable workbench platform for easy deployment and performance evaluation of Federated Learning projects. Through experiments, we demonstrated the ease with which a FL system can be implemented and deployed with FLWB. © 2023 IEEE.

Recent research has demonstrated that Non-Identically Distributed (Non-IID) data can negatively impact the performance of global models constructed in federated learning. To address this concern, multiple approaches have been developed. Nonetheless, previous research lacks a cohesive overview and fails to uniformly assess these strategies, resulting in challenges when comparing and choosing relevant options for real-world scenarios. This study presents a structured survey of cutting-edge techniques for handling the Non-IID data, accompanied by proposing a metric to develop a standardized approach for assessing data skew and its harmony with the appropriate approach. The findings affirm the metric's suitability as a heuristic for assessing data skew in distributed datasets without having insight into client data, serving both scientific and practical purposes and thus supporting the selection of handling strategies. This preliminary research establishes the foundation for discussing standardizing methodologies for evaluating data heterogeneity in federated learning. © 2023 IEEE.

The integration of the Internet of Things (IoT) and Machine Learning (ML) technologies has opened up for the development of novel types of systems and services. Federated Learning (FL) has enabled the systems to collaboratively train their ML models while preserving the privacy of the data collected by their IoT devices and objects. Several FL frameworks have been developed, however, they do not enable FL in open, distributed, and heterogeneous IoT environments. Specifically, they do not support systems that collect similar data to dynamically discover each other, communicate, and negotiate about the training terms (e.g., accuracy, communication latency, and cost). Towards bridging this gap, we propose ART4FL, an end-to-end framework that enables FL in open IoT settings. The framework enables systems’ users to configure agents that participate in FL on their behalf. Those agents negotiate and make commitments (i.e., contractual agreements) to dynamically form federations. To perform FL, the framework deploys the needed services dynamically, monitors the training rounds, and calculates agents’ trust scores based on the established commitments. ART4FL exploits a blockchain network to maintain the trust scores, and it provides those scores to negotiating agents’ during the federations’ formation phase. © 2023 IEEE., Intelligent and Trustworthy IoT Systems

Human Activity Recognition (HAR) plays a significant role in recent years due to its applications in various fields including health care and well-being. Traditional centralized methods reach very high recognition rates, but they incur privacy and scalability issues. Federated learning (FL) is a leading distributed machine learning (ML) paradigm, to train a global model collaboratively on distributed data in a privacy-preserving manner. However, for HAR scenarios, the existing action recognition system mainly focuses on a unified model, i.e. it does not provide users with personalized recognition of activities. Furthermore, the heterogeneity of data across user devices can lead to degraded performance of traditional FL models in the smart applications such as personalized health care. To this end, we propose a novel federated learning model that tries to cope with a statistically heterogeneous federated learning environment by introducing a group-personalized FL (GP-FL) solution. The proposed GP-FL algorithm builds several global ML models, each one trained iteratively on a dynamic group of clients with homogeneous class probability estimations. The performance of the proposed FL scheme is studied and evaluated on real-world HAR data. The evaluation results demonstrate that our approach has advantages in terms of model performance and convergence speed with respect to two baseline FL algorithms used for comparison. © 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.

Federated Learning (FL) provides a promising solution for preserving privacy in learning shared models on distributed devices without sharing local data on a central server. However, most existing work shows that FL incurs high communication costs. To address this challenge, we propose a clustering-based federated solution, entitled Federated Learning via Clustering Optimization (FedCO), which optimizes model aggregation and reduces communication costs. In order to reduce the communication costs, we first divide the participating workers into groups based on the similarity of their model parameters and then select only one representative, the best performing worker, from each group to communicate with the central server. Then, in each successive round, we apply the Silhouette validation technique to check whether each representative is still made tight with its current cluster. If not, the representative is either moved into a more appropriate cluster or forms a cluster singleton. Finally, we use split optimization to update and improve the whole clustering solution. The updated clustering is used to select new cluster representatives. In that way, the proposed FedCO approach updates clusters by repeatedly evaluating and splitting clusters if doing so is necessary to improve the workers’ partitioning. The potential of the proposed method is demonstrated on publicly available datasets and LEAF datasets under the IID and Non-IID data distribution settings. The experimental results indicate that our proposed FedCO approach is superior to the state-of-the-art FL approaches, i.e., FedAvg, FedProx, and CMFL, in reducing communication costs and achieving a better accuracy in both the IID and Non-IID cases. © 2022 by the authors., open access

Breast cancer is one of the most common invading cancers in women. Analyzing breast cancer is nontrivial and may lead to disagreements among experts. Although deep learning methods achieved an excellent performance in classification tasks including breast cancer histopathological images, the existing state-of-the-art methods are computationally expensive and may overfit due to extracting features from in-distribution images. In this paper, our contribution is mainly twofold. First, we perform a short survey on deep-learning-based models for classifying histopathological images to investigate the most popular and optimized training-testing ratios. Our findings reveal that the most popular training-testing ratio for histopathological image classification is 70%: 30%, whereas the best performance (e.g., accuracy) is achieved by using the training-testing ratio of 80%: 20% on an identical dataset. Second, we propose a method named DenTnet to classify breast cancer histopathological images chiefly. DenTnet utilizes the principle of transfer learning to solve the problem of extracting features from the same distribution using DenseNet as a backbone model. The proposed DenTnet method is shown to be superior in comparison to a number of leading deep learning methods in terms of detection accuracy (up to 99.28% on BreaKHis dataset deeming training-testing ratio of 80%: 20%) with good generalization ability and computational speed. The limitation of existing methods including the requirement of high computation and utilization of the same feature distribution is mitigated by dint of the DenTnet. © 2022 Musa Adamu Wakili et al., open access

The use of digital documents is knowing a widespread in different daily administrative and economic transactions. Simultaneously, the forgery of many documents becomes a crime that costs billions to states and companies. Several researchers tried to develop techniques that automatically detect forged documents using machine learning and image processing. With the immense success of deep learning applications, we employ, in this work, a convolutional neural network architecture that uses a gathered dataset of forged and authentic administrative documents. The results obtained on our dataset of 493 documents reached 73.95% accuracy and 97.3% recall, surpassing the efficiency of the machine learning base methods. © 2022 IEEE.

Domain shift is a common problem in many real-world applications using machine learning models. Most of the existing solutions are based on supervised and deep-learning models. This paper proposes a novel clustering algorithm capable of producing an adapted and/or integrated clustering model for the considered domains. Source and target domains are represented by clustering models such that each cluster of a domain models a specific scenario of the studied phenomenon by defining a range of allowable values for each attribute in a given data vector. The proposed domain integration algorithm works in two steps: (i) cross-labeling and (ii) integration. Initially, each clustering model is crossly applied to label the cluster representatives of the other model. These labels are used to determine the correlations between the two models to identify the common clusters for both domains, which must be integrated within the second step. Different features of the proposed algorithm are studied and evaluated on a publicly available human activity recognition (HAR) data set and real-world data from a smart logistics use case provided by an industrial partner. The experiment's goal on the HAR data set is to showcase the algorithm's potential in automatic data labeling. While the conducted experiments on the smart logistics use case evaluate and compare the performance of the integrated and two adapted models in different domains. © 2022 IEEE.

In this paper, we propose a novel method for high-growth firm prediction by minimizing a cost function using a Genetic Algorithm (GA). To achieve it, the GA is used to search to find a set of important variables which provide the best fit for machine learning models so that accurate predictions can be made for high-growth firm prediction. The GA is employed to optimize the mean square error (MSE) between the accurate results and the predicted results of the machine learning methods by evolving the initially generated binary solutions through iterations. The proposed method obtains the best fitting set of variables for the machine learning methods for high-growth firm prediction. Four different machine learning methods which are Support Vector Machines (SVM), Logistic Regression, Random Forest (RF) and K-Nearest Neighbor (K-NN) have been employed with the GA and experimental results show that using RF with the GA achieves the best accuracy results with 94.93%. © 2022 IEEE.

In recent years, deep neural networks (DNNs) achieved state-of-the-art performance on several computer vision tasks. However, the one typical drawback of these DNNs is the requirement of massive labeled data. Even though few-shot learning methods address this problem, they often use techniques such as meta-learning and metric-learning on top of the existing methods. In this work, we address this problem from a neuroscience perspective by proposing a hypothesis named Ikshana, which is supported by several findings in neuroscience. Our hypothesis approximates the refining process of conceptual gist in the human brain while understanding a natural scene/image. While our hypothesis holds no particular novelty in neuroscience, it provides a novel perspective for designing DNNs for vision tasks. By following the Ikshana hypothesis, we design a novel neural-inspired CNN architecture named IkshanaNet. The empirical results demonstrate the effectiveness of our method by outperforming several baselines on the entire and subsets of the Cityscapes and the CamVid semantic segmentation benchmarks. © 2022, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd., open access

Classifying e-mails into distinct labels can have a great impact on customer support. By using machine learning to label e-mails, the system can set up queues containing e-mails of a specific category. This enables support personnel to handle request quicker and more easily by selecting a queue that match their expertise. This study aims to improve a manually defined rule-based algorithm, currently implemented at a large telecom company, by using machine learning. The proposed model should have higher F1-score and classification rate. Integrating or migrating from a manually defined rule-based model to a machine learning model should also reduce the administrative and maintenance work. It should also make the model more flexible. By using the frameworks, TensorFlow, Scikit-learn and Gensim, the authors conduct a number of experiments to test the performance of several common machine learning algorithms, text-representations, word embeddings to investigate how they work together. A long short-term memory network showed best classification performance with an F1-score of 0.91. The authors conclude that long short-term memory networks outperform other non-sequential models such as support vector machines and AdaBoost when predicting labels for e-mails. Further, the study also presents a Web-based interface that were implemented around the LSTM network, which can classify e-mails into 33 different labels. © 2020, The Author(s)., Open access

Sentiment analysis using stemmed Twitter data from various languages is an emerging research topic. In this paper, we address three data augmentation techniques namely Shift, Shuffle, and Hybrid to increase the size of the training data; and then we use three key types of deep learning (DL) models namely recurrent neural network (RNN), convolution neural network (CNN), and hierarchical attention network (HAN) to classify the stemmed Turkish Twitter data for sentiment analysis. The performance of these DL models has been compared with the existing traditional machine learning (TML) models. The performance of TML models has been affected negatively by the stemmed data, but the performance of DL models has been improved greatly with the utilization of the augmentation techniques. Based on the simulation, experimental, and statistical results analysis deeming identical datasets, it has been concluded that the TML models outperform the DL models with respect to both training-time (TTM) and runtime (RTM) complexities of the algorithms; but the DL models outperform the TML models with respect to the most important performance factors as well as the average performance rankings. CCBY, open access

In this paper, forecasting sales model for truck components using machine learning algorithms is proposed. The forecasting model helps companions (i.e. Volvo Trucks) in the activity of trade and business. It also plays a major role for firms in decision-making operations in the areas corresponding to sales, production, purchasing, finance, and accounting. In order to achieve good forecasting sales mode, firstly, a normalization approach is performed on the time-series data to reduce and eliminate the data redundancy. After that, feature extraction and selection techniques are employed on the normalized data. Finally, different machine learning methods such as Support Vector Machine Regression, Ridge Regression, Gradient Boosting Regression and Random Forest Regression have been applied to the features of the normalized time-series data. Results depict that ridge regression method gives the most promising forecasting sale results of truck components compared to the other machine learning methods. © 2020 IEEE., open access

One of the crucial aspects of additive manufacturing is the monitoring of the welding process for quality assurance of components. A common way to analyse the welding process is through visual inspection of melt-pool images to identify possible defects in manufacturing. Recent literature studies showed the potential use of prediction models for defects classification to speed up the manual verification criteria since a huge data is generated from the additive manufacturing. Although a huge image data is available, the data needs to be labelled manually by experts which results in small sample datasets. Hence, to model small sample sizes and also to acquire the importance of parameters, we opted a traditional machine learning method, Random Forests (RF). For feature extraction, we opted for the Polar Transformation to explore its applicability using the melt-pool image dataset and a publicly available shape image dataset. The results show that RF models with Polar Transformation performed the best on our case study datasets and the second-best for the public dataset when compared to the Histogram of Oriented Gradients, HARALICK, XY-projections of an image, and Local Binary Patterns methods. As such, the Polar Transformation can be considered as a suitable compact shape descriptor. © 2020 IEEE., open access