Search

The complexity surrounding the seamless execution of skilled action sequences during sparse rewards necessitates an organization of behavior into modular, temporally extended sub-policies. The rodent Medial Prefrontal Cortex (mPFC) has emerged as a critical brain region implicated in orchestrating such action plans over extended time frames. Previous research has shed light on the mPFC’s mechanistic contributions to the control of behavior through investigations into its numerous attributed roles, encompassing working memory, spatial representation, decision-making, and task-specific generalization. We investigated the neural correlates of position, task-specific features, and actions within the rat mPFC during a behaviorally elaborate, reward-guided sequence navigation paradigm. We identified a significant proportion of mPFC neurons exhibiting a robust task-specific code for tracking animal’s progress between rewards, characterized most prominently by modulation departures from and approaches to reward zones. Furthermore, initial population-level analyses suggested a representation of prospective non-immediate actions, encoded as turn directions, extending beyond immediate temporal horizons. However, subsequent in-depth examination revealed that these initial findings could not hold up to predict future actions at single-trial level. Instead, the apparent prospective coding of turn directions seemed to be confounded by preparatory motor behaviors within the navigational environment. Consequently, these results call for a reassessment of the influence of various sensorimotor factors on mPFC activity during goal-directed decision-making, challenging the prevailing methodologies used to infer cognitive processes within this region.

The Internet of Things (IoT) is rapidly expanding, connecting an increasing number of devices daily. Having diverse and extensive networking and resource-constrained devices creates vulnerabilities to various cyber-attacks. The IoT with the supervision of Software Defined Network (SDN) enhances the network performance through its flexibility and adaptability. Different methods have been employed for detecting security attacks; however, they are often computationally efficient and unsuitable for such resource-constraint environments. Consequently, there is a significant requirement to develop efficient security measures against a range of attacks. Recent advancements in deep learning (DL) models have paved the way for designing effective attack detection methods. In this study, we leverage Genetic Algorithm (GA) with a correlation coefficient as a fitness function for feature selection. Additionally, mutual information (MI) is applied for feature ranking to measure their dependency on the target variable. The selected optimal features were used to train a hybrid DNN model to uncover attacks in IoT networks. The hybrid DNN integrates Convolutional Neural Network, Bi-Gated Recurrent Units (Bi-GRU), and Bidirectional Long Short-Term Memory (Bi-LSTM) for training the input data. The performance of our proposed model is evaluated against several other baseline DL models, and an ablation study is provided. Three key datasets InSDN, UNSW-NB15, and CICIoT 2023 datasets, containing various types of attacks, were used to assess the performance of the model. The proposed model demonstrates an impressive accuracy and detection time over the existing model with lower resource consumption.

Taking backorders on products is a common scenario in inventory and supply chain management systems. The ability to predict the likelihood of backorders can surely minimise a company's losses. Because the number of backorders is much lower than the number of orders that ship on time, applying a predictive model for this domain is a challenging task. This paper proposes a model that uses a deep neural network to predict backorders; it handles the data imbalance between backorders and filled orders with efficient techniques. To make the dataset balanced, we employ different techniques that include minority class weight boosting, randomised oversampling, SMOTE oversampling, and a combination of oversampling and undersampling. The balanced training data are used in our proposed, fully connected deep neural networks model to train the predictive model. The predictive model learns the likelihood of product backorders by using the training samples. We conduct experiments on a large benchmark dataset to test the performance of our proposed deep neural network-based model. The experimental results achieve a new state-of-the-art performance and outperform some prominent classification models in terms of standard evaluation metrics and expected profit measure., Přijímání zpětných objednávek na produkty je běžným scénářem v systémech řízení zásob a dodavatelského řetězce. Schopnost předvídat pravděpodobnost zpětných objednávek může jistě minimalizovat ztráty společnosti. Protože počet zpětných objednávek je mnohem nižší než počet objednávek, které jsou odeslány včas, je použití predikčního modelu pro tuto oblast náročným úkolem. Tento článek navrhuje model, který k předpovídání zpětných objednávek využívá hlubokou neuronovou síť; pomocí účinných technik se vypořádává s nerovnováhou dat mezi zpětnými objednávkami a vyplněnými objednávkami. Aby byl soubor dat vyvážený, používáme různé techniky, které zahrnují posilování váhy menšinových tříd, náhodné převzorkování, převzorkování SMOTE a kombinaci převzorkování a podvzorkování. Vyvážená trénovací data jsou použita v námi navrženém, plně propojeném modelu hlubokých neuronových sítí k trénování predikčního modelu. Prediktivní model se pomocí trénovacích vzorků učí pravděpodobnost zpětných objednávek produktů. Provádíme experimenty na velkém souboru referenčních dat, abychom otestovali výkonnost námi navrženého modelu založeného na hlubokých neuronových sítích. Výsledky experimentů dosahují nového špičkového výkonu a překonávají některé významné klasifikační modely z hlediska standardních hodnotících metrik a míry očekávaného zisku.

As electric vehicles (EV) become increasingly popular in the automotive world, an accurate State-of-Charge (SoC) estimation is critical to optimizing energy utilization, increasing driving range and ensuring long-lasting battery system. This research focuses on the application of Deep Neural Networks (DNN) as an SoC estimation method in EV, exploiting the inherent capacity of DNN to learn complex relationships in vast data sets. The results of the performed simulations show that the proposed DNN-based SoC estimation method achieves a high level of accuracy, outperforming traditional estimation techniques, especially in scenarios involving rapid changes in driving conditions. This research also explores the impact of Neural Networks architecture and hyperparameter tuning on overall performance and provides insights for optimizing DNN-based SoC estimation systems. From the tests that have been carried out, an error value of 1.3% is obtained from the results of the training carried out on the DNN structure that has been prepared., Seiring dengan semakin populernya kendaraan listrik (EV) di dunia otomotif, estimasi State-of-Charge (SoC) yang akurat sangat penting untuk mengoptimalkan pemanfaatan energi, meningkatkan jangkauan berkendara dan memastikan sistem baterai tahan lama. Penelitian ini berfokus pada penerapan Deep Neural Networks (DNN) sebagai metode estimasi SoC pada EV, memanfaatkan kapasitas bawaan DNN untuk mempelajari hubungan kompleks dalam kumpulan data yang luas. Hasil dari simulasi yang dilakukan menunjukkan bahwa metode estimasi SoC berbasis DNN yang diusulkan mencapai tingkat akurasi yang tinggi, mengungguli teknik estimasi tradisional, terutama dalam skenario yang melibatkan perubahan cepat dalam kondisi berkendara. Penelitian ini juga mengeskplorasi dampak arsitektur Neural Networks dan penyetelan hyperparameter terhadap kinerja secara keseluruhan, serta memberikan wawasan untuk mengoptimalkan sistem estimasi SoC berbasis DNN. Dari pengujian yang telah dilakukan didapat nilai error sebesar 1.3% dari hasil training yang dilakukan terhadap struktur DNN yang telah disusun.

As training Deep Neural Network (DNN) is time-consuming, people resort to parallelization across multiple accelerators. A plethora of solutions adopt data/model parallelization, but they suffer from frequent weight synchronization overhead or resource under-utilization. Recent work introduces pipeline parallelism to improve the utilization of accelerators, however, most existing pipeline parallelism approaches take a one-shot configuration, while ignoring the fluctuation of available resources, e.g., bandwidth and GPUs. Moreover, the heuristic work partition methods oversimplify the computation and communication process, leading to sub-optimal results. To address this challenge, we present AutoPipe, a self-adaptive pipeline parallelism optimization solution. At its core, AutoPipe introduces a reinforcement learning (RL) based work partitioning model, which takes into account both exact communication procedure and dynamic state switching. To mitigate the stalls on state switching, AutoPipe adopts layer-by-layer computation under switching. We have implemented an AutoPipe prototype and evaluated it via testbed experiments. Our results show that the AutoPipe-enhanced PipeDream can find better work partitioning and benefit from dynamic configuration, outperforming the vanilla solutions by up to 89% for exclusive tasks and 143% in dynamic workloads. Furthermore, we show that AutoPipe can also work well with other pipeline parallelism schemes and achieve considerable performance gains. © 2024 Owner/Author.

This paper examines the features of constructing a physical model of an unmanned aerial vehicle (UAV) for computer vision tasks. The study analyzes existing physical models, highlighting their strengths and weaknesses. Consequently, it describes existing methods and tools for route planning and navigation. Moreover, the research outlines preliminary achievements in reducing the size of neural network models for computer vision tasks, specifically linear quantization of neural networks and half-precision computation. We identify approaches and principles for developing an autonomous control system for UAVs. As a result, we develop the structure of the UAVs autonomous control system and note the characteristics of its computer tools. For example, the concept for constructing a geodata correction algorithm uses images from a camera, satellite images, inertial measurement units, and a global positioning system (GPS). The study improves the method of neural network object recognition by parallelizing video data processing using multiple digital accelerators. The benefits of this study include proposed scenarios for using the system with numerous hardware accelerators. We describe the performance of models before and after quantization in the context of recognition accuracy. Furthermore, we present a method of partial quantization for processing ultra-large neural networks in computer vision. The proposed methods and technologies demonstrate high potential for improving the efficiency and accuracy of data processing under real-world operating conditions. Improved approaches to neural network quantization and optimization significantly reduce model size and increase processing speed while maintaining high recognition accuracy. Consequently, the integration of multiple digital accelerators allows for effective parallelization of video data processing, which is particularly important for real-time information analysis. The results of this study can be used to improve navigat

Deep neural networks have demonstrated significant achievements across various fields, yet their memory and time complexities present obstacles for implementing them on resource-constrained devices. Compressing deep neural networks using tensor decomposition can decrease both memory usage and computational costs. The performance of a low-rank tensorized network depends on the choices of hyperparameters including the tensor rank and geometry. Previous studies have concentrated on identifying optimal tensor ranks. This thesis studies the effect of tensor geometry used for folding data for low-rank tensor compression. It is demonstrated that tensor geometry significantly affects compression efficiency of the tensorized data and model parameters. Consequently, a novel mathematical formulation is developed to optimize tensor geometry. The tensor geometry optimization model is adopted for efficient deployment of low-rank neural networks. The presented tensor geometry optimization model is combinatorial and thus challenging to solve. Therefore, surrogate and relaxed versions of the model are developed and various methods including integer linear programming, graph optimization, and random search algorithms are applied to solve the presented optimization model. The proposed tensor geometry optimization achieved a notable reduction in both the memory and time complexities of neural networks while maintaining accuracy. The developed methods can be applied for hardware-software co-design of artificial intelligence (AI) accelerators particularly on resource-constrained devices.

Deep learning has made remarkable progress over the past decade across various fields, such as Computer Vision, Natural Language Processing (NLP), and Speech Recognition, driving innovation and advancements of various applications. One key challenge is to improve deep learning models' generalization, which refers to the capability of a model to handle unseen data. Several factors contribute to the challenge of generalization in deep learning models, including limited data availability, overfitting tendencies, and the inherent complexity of the models themselves. The phenomenon of adversarial samples is symptomatic of the limitation in the generalization capability of deep learning models. Adversarial samples refer to data instances manipulated from their originals, often appearing visually similar and imperceptible to human senses, yet causing incorrect predictions from deep learning models. This phenomenon presents a substantial concern for security-sensitive domains like medical diagnosis, autonomous driving, and anomaly detection, where model reliability is crucial. The development of various adversarial defense methods in recent years, such as adversarial training, noise reduction, and gradient masking, emphasizes the considerable efforts to enhance the robustness and reliability of deep learning models. Meanwhile, as innovative adversarial attacks continue to evolve, they effectively expose the vulnerabilities inherent in deep learning models, thereby raising challenges for existing defense methodologies. Although adversarial defense research has made advancements, the root cause of the vulnerability in deep learning models is still not fully understood. Additionally, there is a pressing need for defense mechanisms that offer comprehensive protection and high resilience against a wide range of adversarial attacks. The research presented in this dissertation aims to contribute to the enrichment of knowledge within the research community by providing deeper insights into adversarial attacks and defense mechanisms. It endeavors to develop novel defense methods that are robust and reliable when protecting deep learning models against adversarial threats. Through this work, we seek to advance the field of adversarial defense and contribute to the development of more effective defense strategies. In this dissertation, we introduced three novel defense mechanisms aimed at enhancing the robustness of deep learning models against adversarial attacks. In the first work, we tackled the issue of image blurring in traditional Variational Autoencoder (VAE)-based generative networks by focusing on improving high-fidelity data reconstruction. Additionally, this work optimized the model's decision-making strategy through a Bayesian update, allowing a model to incorporate multiple sources of supporting evidence for the final decision. The second study proposed a new generative network structure coupled with a new two-step noise reduction approach designed to effectively filter out adversarial noise. The third method introduced a new noise reduction mechanism called VQUNet. This method features a learnable quantization of latent features and a hierarchical network structure for high-fidelity data reconstruction. VQUNet's unique design significantly enhances the data reconstruction quality after the filtering process, while effectively regularizing adversarial perturbation within the network, thereby improving its resilience against adversarial attacks. Extensive experimental investigations demonstrated that the proposed methods provided superior robustness to the targeted deep learning models. They exhibited superior performance over other state-of-the-art noise-reduction-based defense methods, achieving prediction improvement with a notable margin over existing methods under adversarial attacks across both Fashion-MNIST and CIFAR10 datasets. The experimental analysis underscored the effectiveness, resilience, and robustness of the proposed methods against adversarial attacks. These findings offered valuable insights into the development of effective defense strategies, shedding light on the mechanisms and principles for future research.

The current technological revolution driven by advances in machine learning has motivated a wide range of applications aiming to improve our quality of life. Representative of such applications are autonomous and semiautonomous Powered Wheelchairs (PWs), where the focus is on providing a degree of autonomy to the wheelchair user as a matter of guidance and interaction with the environment. Based on these perspectives, the focus of the current research has been on the design of lightweight systems that provide the necessary accuracy in the navigation system while enabling an embedded implementation. This motivated us to develop a real-time measurement methodology that relies on a monocular RGB camera to detect the caregiver’s feet based on a deep learning method, followed by the distance measurement of the caregiver from the PW. An important contribution of this article is the metrological characterization of the proposed methodology in comparison with measurements made with dedicated depth cameras. Our results show that despite shifting from 3D imaging to 2D imaging, we can still obtain comparable metrological performances in distance estimation as compared with Light Detection and Ranging (LiDAR) or even improved compared with stereo cameras. In particular, we obtained comparable instrument classes with LiDAR and stereo cameras, with measurement uncertainties within a magnitude of 10 cm. This is further complemented by the significant reduction in data volume and object detection complexity, thus facilitating its deployment, primarily due to the reduced complexity of initial calibration, positioning, and deployment compared with three-dimensional segmentation algorithms.

This paper presents a hardware inference accelerator architecture of quantized deep neural networks (DNN). The proposed accelerator implements all computation in a quantize version of DNN including linear transformations like matrix multiplications, nonlinear activation functions such as ReLU, quantization and dequantization operation. The hardware accelerator of quantized DNN consists of matrix multiplication core which is implemented in systolic array architecture, and the QDR core for computing the operation of quantization, dequantization, and ReLU. This proposed hardware architecture is implemented in Verilog Hardware Description Language (HDL) code using modelsim. To validate, we simulated the quantized DNN using Python programming language and compared the results with our proposed hardware accelerator. The result of this comparison shows a very slight difference, confirming the validity of our quantized DNN hardware accelerator.

A force plate is mainly used in biomechanics; it aims to measure the ground reaction force in a persons walking or standing position. In this study, a large-area force mat of the piezoresistance sensing type was developed, and a deep-learning-based weight measurement calibration method was applied to solve the problem in which measurements are not normalized because of physical limitations in hardware and signal processing. The test set was composed of the values measured at each point by weight and the value of the center of the pressure variable, and the measured value was predicted using a deep neural network (DNN) regression model. The calibration verification results show that the average weight errors range from a minimum of 0.06% to a maximum of 3.334%. This is simpler than the previous method, which directly measures the ratio of the resistance value to the measured weight of each sensor and derives an equation.

This study delves into the potential of contactless palmprints within large-scale biometric frameworks, focusing on improving candidate narrowing through an encoder-based approach. Utilizing deep neural networks and trained via semi-hard triplet learning, the research transforms palm images into distinctive feature vectors for precise identification and candidate selection. Comprehensive analysis involving various architectures, datasets, and preprocessing techniques achieved a closed-set rank 10 retrieval rate of 99.4\% on the HandID and Tongji datasets. Additionally, the Average Number of Hands (ANH) metric was introduced for model comparison, revealing that Model 62 outperformed others across multiple tests. Although the models are not yet sufficient as standalone end-to-end classifiers, they exhibit strong potential when combined with additional classifiers. Comparisons to previous studies underscore the promising performance of palmprint biometrics, highlighting their potential in specific domains like access security and payment.

The success of deep learning on image classification and recognition tasks has led to new applications in diverse contexts, including the field of medical imaging. However, two properties of deep neural networks (DNNs) may limit their future use in medical applications. The first is that DNNs require a large amount of labeled training data, and the second is that the deep learning-based models lack interpretability. In this paper, we propose and investigate a data-efficient framework for the task of general medical image segmentation. We address the two aforementioned challenges by introducing domain knowledge in the form of a strong prior into a deep learning framework. This prior is expressed by a customized dynamical system. We performed experiments on two different datasets, namely JSRT and ISIC2016 (heart and lungs segmentation on chest X-ray images and skin lesion segmentation on dermoscopy images). We have achieved competitive results using the same amount of training data compared to the state-of-the-art methods. More importantly, we demonstrate that our framework is extremely data-efficient, and it can achieve reliable results using extremely limited training data. Furthermore, the proposed method is rotationally invariant and insensitive to initialization. © 2024 The Authors

In time-division duplex (TDD) distributed large-scale multiple input multiple output (DM-MIMO) systems, the traditional downlink channel precoding method is used to resist inter-user interference (IUI). However, when the Channel State Information (CSI) is incomplete, the performance loss is serious, not only the bit error rate is high, but also the complexity of the traditional precoding algorithm is high. In order to solve these problems, this paper proposes an adaptive precoding framework based on deep learning (DL) for joint training and split application deployment. First, we train a channel emulator deep neural network (CE-DNN) to learn and simulate the transmission process of the wireless communication channel. Then, we concatenate an untrained precoding DNN (P-DNN) with a trained CE-DNN and retrain the cascaded neural network to converge. The last step is to obtain the P-DNN, namely the adaptive precoding network, by dismantling the joint trained network. Simulation results show that, when CSI is imperfect, the proposed method is compared with Tomlinson-Harashima precoding (THP) and block diagonalization (BD) precoding. The proposed method has a lower mean square error (MSE) and higher spectrum efficiency, as well as a bit error rate (BER) performance close to the THP. The source codes and the neural network codes are available on request.

The building sector in Sweden is responsible for 40 % of the final energy use, and space heating largely contributes to the total energy demand. Minimum energy performance standards and building codes are becoming more strict and comprehensive globally. In the European Union there are substantial goals of achieving a higher energy efficiency in the residential building sector. Energy Performance Certificates (EPCs) are a tool that can be used to track and assess building energy in a standardized way, to follow up on the energy efficiency of buildings. Privately owned single-family houses do not need to have a valid EPC, it is only mandatory to perform an assessment when the house is subject to sale, meaning that only about 20 % of single-family houses in Sweden have an EPC. The objective of the study is therefore to develop a data driven model based on existing EPC data, to estimate the annual space heating energy demand of Swedish single family houses that do not have an EPC. Sub-objectives include testing different data-driven machine learning models to identify which algorithm achieves the highest accuracy. Input features such as floor area, construction year, geographical location and heating source were used to make predictions. The results show that out of the investigated models, the Histogram-GBM performed the best when predicting the space heating energy of an individual house, with a RMSE of 24.33 kWh/m2 /year, MAE of 17.50 kWh/m2 /year, MAPE of 24.62 % and R2 -value of 0.736. When evaluating the results of predictions for the entire single-family housing stock, the errors of each individual house are hidden within the entire housing stock, minimizing the error of the total prediction. The best performing model, Histogram GBM, achieved a MBE of 0.59 kWh/prediction, equivalent to an error smaller than 0.005 %., I Sverige står byggnadssektorn för 40 % av den slutliga energianvändningen, och uppvärmning av byggnader är en stor andel av den totala energianvändningen. Minimikrav för energiprestanda och byggregler blir globalt allt mer strikta och omfattande. EU har betydande mål att uppnå högre energieffektivitet för alla typer av bostadshus. Energideklarationer (EPC) är ett verktyg som kan användas för att samla in och bedömma energianvändning i byggnader på ett standardiserat sätt, för att kunna följa upp energieffektivisering av byggnader. Till skillnad från andra byggnadstyper behöver privat ägda småhus inte ha en giltig energideklaration, de är endast obligatoriska att utfärda när huset säljs, vilket betyder att bara 20 % av småhus i Sverige har en energideklaration. Syftet med studien är därför att utveckla en datadriven modell baserat på data från existerande energideklarationer, för att estimera den årliga värmeenergianvändningen i svenska småhus som saknar en energideklaration. Syftet är också att testa olika data-drivna maskininlärningsmodeller för att identifiera vilken som uppnår det lägsta felet. I modellerna har indata såsom golvyta, byggår, geografisk plats och värmekälla använts för att göra estimeringar. Resultaten visar att av de undersökta modellerna var Histogram-GBM den bästa för att estimera värmeanvändning, med ett RMSE på 24.33 kWh/m2 /år, MAE på 17.50 kWh/m2 /år, MAPE på 24.62 % och ett R2 -värde på 0.736. Resultaten visar också att när värmeanvändning estimeras för hela beståndet av småhus, minimeras det totala felet när alla kvarvarande fel summeras. Den bästa modellen, Histogram-GBM uppnådde ett MBE på 0.59 kWh/estimering, vilket motsvarar ett fel mindre än 0.005 %.

This project focuses on deep neural network (DNN)-based non-intrusive speech quality assessment, specifically addressing the challenge of predicting mean-opinion-score (MOS) with interpretable posterior distributions. The conventional approach of providing a single point estimate for MOS lacks interpretability and doesn't capture the uncertainty inherent in subjective assessments. This thesis introduces DeePMOS, a novel framework capable of producing MOS predictions in the form of posterior distributions, offering a more nuanced and understandable representation of speech quality. DeePMOS adopts a CNN-BLSTM architecture with multiple prediction heads to model Gaussian and Beta posterior distributions. For robust training, we use a combination of maximum-likelihood learning, stochastic gradient noise, and a student-teacher learning setup to handle limited and noisy training data. Results showcase DeePMOS's competitive performance, particularly with DeePMOS-B achieving state-of-the-art utterance-level performance. The significance lies in providing accurate predictions along with a measure of confidence, enhancing transparency and reliability. This opens avenues for application in domains such as telecommunications and audio-processing systems. Future work could explore additional posterior distributions, evaluate the model on high-quality datasets, and consider incorporating listener-dependent scores., Detta projekt fokuserar på icke-intrusiv bedömning av tal-kvalitet med hjälp av djupa neurala nätverk (DNN), särskilt för att hantera utmaningen att förutsäga mean-opinion-score (MOS) med tolkningsbara posteriora fördelningar. Den konventionella metoden att ge en enda punktsuppskattning för MOS saknar tolkningsbarhet och fångar inte osäkerheten som är inneboende i subjektiva bedömningar. Denna avhandling introducerar DeePMOS, en ny ramverk kapabel att producera MOS-förutsägelser i form av posteriora fördelningar, vilket ger en mer nyanserad och förståelig representation av tal-kvalitet. DeePMOS antar en CNN-BLSTM-arkitektur med flera förutsägelsehuvuden för att modellera Gaussiska och Beta-posteriora fördelningar. För robust träning använder vi en kombination av maximum-likelihood learning, stokastisk gradientbrus och en student-lärare inlärningsuppsättning för att hantera begränsad och brusig träningsdata. Resultaten visar DeePMOS konkurrenskraftiga prestanda, särskilt DeePMOS-B som uppnår state-of-the-art prestanda på uttalnivå. Signifikansen ligger i att ge noggranna förutsägelser tillsammans med en mått på förtroende, vilket ökar transparensen och tillförlitligheten. Detta öppnar möjligheter för tillämpningar inom områden som telekommunikation och ljudbehandlingssystem. Framtida arbete kan utforska ytterligare posteriora fördelningar, utvärdera modellen på högkvalitativa dataset och överväga att inkludera lyssnarberoende poäng.

Colorectal cancer is one of the most prevalent malignancies globally and recently introduced digital pathology enables the use of machine learning as an aid for fast diagnostics. This project aimed to develop a deep neural network model to specifically identify and differentiate dysplasia in the epithelium of colorectal polyps and was posed as a binary classification problem. The available dataset consisted of 80 whole slide images of different H&E-stained polyp sections, which were parted info smaller patches, annotated by a pathologist. The best performing model was a pre-trained ResNet-18 utilising a weighted sampler, weight decay and augmentation during fine tuning. Reaching an area under precision-recall curve of 0.9989 and 97.41% accuracy on previously unseen data, the model’s performance was determined to underperform compared to the task’s intra-observer variability and be in alignment with the inter-observer variability. Final model made publicly available at https://github.com/stinabr/classification-of-colorectal-polyps.

This master thesis focuses on applying deep neural networks (DNNs) in image-based condition monitoring of air-jet spinning machines, specifically focusing on the spinning pressure parameter. The study aims to develop a sensor system to detect structural defects in yarns and assign them to specific machine conditions. The research explores using DNNs to analyze images of yarns generated at different spinning pressures within the spinning box to create a rich dataset for training deep learning models. The study also evaluates the effectiveness of the DNN-based approach in detecting and classifying structural defects in yarns and determining the corresponding machine conditions. The outcomes of this research could potentially help textile enterprises improve the quality and efficiency of their yarn manufacturing processes.

This paper investigates the effects of various randomization techniques on Deep Neural Networks (DNNs) learning performance. We categorize the existing randomness techniques into four key types: injection of noise/randomness at the data, model structure, optimization or learning stage. We use this classification to identify gaps in the current coverage of potential mechanisms for the introduction of randomness, leading to proposing two new techniques: adding noise to the loss function and random masking of the gradient updates. We use a Particle Swarm Optimizer (PSO) for hyperparameter optimization and evaluate over 30,000 configurations across standard computer vision benchmarks. Our study reveals that data augmentation and weight initialization randomness significantly improve performance, and different optimizers prefer distinct randomization types. The complete implementation and dataset are available on GitHub1. This paper for the Hot-off-the-Press track at GECCO 2024 summarizes the original work published at [2]. © 2024 Copyright held by the owner/author(s). [2] Mohammed Ghaith Altarabichi, Sławomir Nowaczyk, Sepideh Pashami, Peyman Sheikholharam Mashhadi, and Julia Handl. 2024. Rolling the dice for better deep learning performance: A study of randomness techniques in deep neural networks. Information Sciences 667 (2024), 120500.

Deep reinforcement learning (DRL) is a promising solution for voltage control of distribution grids with high penetration of inverter-based renewable energy sources (RESs). Yet, when adopting the DRL-based control method, the safe and optimal operation of the system cannot be guaranteed at the same time, as the conventional DRL agent is not designed to solve the hard constraint problem. To address this challenge, this paper proposes a deep neural network (DNN) assisted projection based DRL method for safe control of distribution grids. First, a finite iteration projection algorithm is proposed to guarantee hard constraints by converting a non-convex optimization problem into a finite iteration problem. Next, a DNN assisted projection method is proposed to accelerate the calculation of projection and achieve the practical implementation of hard constraints in DRL problem. Finally, a DNN Projection embedded twin-delayed deep deterministic policy gradient (DPe-TD3) method is proposed to achieve optimal operation of distribution grids with guaranteed 100% safety of the distribution grid. The safety of the DRL training is guaranteed via the embedded Projection DNN in TD3 with participation in gradient return process, which could smoothly and effectively project the DRL agent actions into the feasible area, thus guaranteeing the safety of data driven control and the optimal operation at the same time. The case studies and comparisons are conducted in the IEEE 33 bus system to show the effectiveness of the proposed method., QC 20240709

One advantage of photon-counting CT compared to dual-energy CT is the possibility to perform K-edge imaging, where contrast agents such as iodine can be distinguished from other substances based on spectral characteristics. However, for iodine K-edge imaging in clinical CT, the three-basis decomposition problem is ill-conditioned due to the low K-edge energy of iodine, meaning that the decomposition is highly sensitive to both noise and miscalibrations. This makes robust three-basis decomposition difficult using standard techniques. In this simulation study we evaluate a novel method of performing K-edge imaging, which circumvents the challenging three-basis decomposition step by replacing it with multiple two-basis decompositions followed by a deep convolutional neural network to generate three basis images. Based on the XCAT phantom, we generated 1224 spectral phantom image slices of the neck, with iodine-filled blood vessels and calcifications, and simulated CT imaging in CatSim with a silicon-based detector model without quantum noise, i.e. in the high-dose limit. For each simulated slice, we used maximum likelihood to perform three two-basis decompositions, into PE-PVC, PE-iodine, and PVC-iodine, yielding six basis images in total. We then trained a U-Net to map these six input images to the ground-truth basis images, PE, PVC and iodine. Our results show that the proposed method can reproduce PE, PVC and iodine basis images with high accuracy, in the high-dose limit. This suggests that the proposed three-basis decomposition method may be a feasible way of performing K-edge CT imaging with iodine, with important potential implications for imaging of the carotid arteries., QC 20240605Part of ISBN 978-151067154-6

The wide integration of inverter based renewable energy sources (RESs) in modern grids may cause severe voltage violation issues due to high stochastic fluctuations of RESs. Existing centralized approaches can achieve optimal results for voltage regulation, but they have high communication burdens; existing decentralized methods only require local information, but they cannot achieve optimal results. Deep reinforcement learning (DRL) based methods are effective to deal with uncertainties, but it is difficult to guarantee secure constraints in existing DRL training. To address the above challenges, this paper proposes a projection embedded multi-agent DRL algorithm to achieve decentralized optimal control of distribution grids with guaranteed 100% safety. The safety of the DRL training is guaranteed via an embedded safe policy projection, which could smoothly and effectively restrict the DRL agent action space, and avoid any violation of physical constraints in distribution grid operations. The multi-agent implementation of the proposed algorithm enables the optimal solution achieved in a decentralized manner that does not require real-time communication for practical deployment. The proposed method is tested in modified IEEE 33-bus distribution and compared with existing methods; the results validate the effectiveness of the proposed method in achieving decentralized optimal control with guaranteed 100% safety and without the requirement of real-time communications., QC 20240429

With emergence of new materials, more and more materials are available for adsorption and separation processes. The adsorption selectivity of adsorbent to adsorbate is one of the important indicators in choosing materials. Because the adsorption experiment of the mixture is time-consuming and difficult, the selectivity of the adsorbent is generally calculated by the ideal adsorbed solution theory (IAST). Taking the CO2/H2 gas mixture as an example, this paper proposes a new adsorption selectivity calculation method based on a deep neural network (DNN) with 5 hidden layers, which takes the molar fraction of CO2, adsorption pressure and Langmuir adsorption isotherm parameters as the inputs of DNN. Combining the DNN and the NIST/ARPA-E database to quickly and accurately calculate the adsorption selectivity, the hydrogen purification and carbon dioxide storage materials can be quickly screened.

The feature to learn complex text representations enabled by Deep Neural Networks (DNNs) has revolutionized Natural Language Processing and several other fields. However, DNNs have not developed beyond all challenges. For instance, the vanishing gradient problem remains a major challenge. This challenge hinders the ability of the system to capture long-term dependencies in text data. This challenge limits the ability to understand context, implied meanings, semantics, and to represent intricate patterns in text. This study aims to address the prevalent vanishing gradient problem encountered in DNNs when dealing with text data. Text data’s inherent sparsity and heterogeneity exacerbate this issue, increasing computational complexities and processing time. To tackle this problem comprehensively, we will explore existing literature and conduct a bibliometric analysis to identify potential solutions. The findings will contribute to a comprehensive review of the existing literature and suggest effective strategies for mitigating the vanishing gradient problem in the context of NLP tasks. Ultimately, our study will pave the way for further advancements in this area of research.

The Internet of Things (IoT) is currently seeing tremendous growth due to new technologies and big data. Research in the field of IoT security is an emerging topic. IoT networks are becoming more vulnerable to new assaults as a result of the growth in devices and the production of massive data. In order to recognize the attacks, an intrusion detection system is required. In this work, we suggested a Deep Learning (DL) model for intrusion detection to categorize various attacks in the dataset. We used a filter-based approach to pick out the most important aspects and limit the number of features, and we built two different deep-learning models for intrusion detection. For model training and testing, we used two publicly accessible datasets, NSL-KDD and UNSW-NB 15. First, we applied the dataset on the Deep neural network (DNN) model and then the same dataset on Convolution Neural Network (CNN) model. For both datasets, the DL model had a better accuracy rate. Because DL models are opaque and challenging to comprehend, we applied the idea of explainable Artificial Intelligence (AI) to provide a model explanation. To increase confidence in the DNN model, we applied the explainable AI (XAI) Local Interpretable Model-agnostic Explanations (LIME ) method, and for better understanding, we also applied Shapley Additive Explanations (SHAP)., Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public., Cyber Security

Nanopore sequencing generates noisy electrical signals that need to be converted into a standard string of DNA nucleotide bases using a computational step called basecalling. The performance of basecalling has critical implications for all later steps in genome analysis. Therefore, there is a need to reduce the computation and memory cost of basecalling while maintaining accuracy. We present RUBICON, a framework to develop efficient hardware-optimized basecallers. We demonstrate the effectiveness of RUBICON by developing RUBICALL, the first hardware-optimized mixed-precision basecaller that performs efficient basecalling, outperforming the state-of-the-art basecallers. We believe RUBICON offers a promising path to develop future hardware-optimized basecallers.

This work proposes an information security vehicleto- grid (V2G) scheduling solution, which combines Federated deep learning with distributed edge computing for V2G operation. In this framework, each charging point is equipped with an intelligent computing module to conduct distributed edge scheduling for the connected electric vehicle (EV), so that not only the computation of inference process is efficient, but also the privacy-preserving of EV users is guaranteed. Besides, the desensitized V2G data of charging points are used to train the deep neural network model in each charging station. Therefore, the accurate future data acquisition problem and the uncertainty handling challenges under traditional optimization methods is avoided. At the same time, the spatial-based and time-based clustering methods are applied to improve the accuracy of prediction. Finally, through federated learning, each charging station uploads the local model to the cloud server, and a stochastic client selection pattern is designed to improve the scalability of model aggregation in the cloud server. In this way, the digital assets of each charging station are protected, and computing and communication costs are reduced. Simulation results on real datasets show that the proposed framework has superior performance in terms of training accuracy, communication burden, and computing performance, while maintaining the privacy of EV users and the digital assets of charging stations. IEEE

Traffic congestion is one of the growing urban problem with associated problems like fuel wastage, loss of lives, and slow productivity. The existing traffic system uses programming logic control (PLC) with round-robin scheduling algorithm. Recent works have proposed IoT-based frameworks that use traffic density of each lane to control traffic movement, but they suffer from low accuracy due to lack of emergency vehicle image datasets for training deep neural networks. In this paper, we propose a novel distributed IoT framework that is based on two observations. The first observation is major structural changes to road are rare. This observation is exploited by proposing a novel two stage vehicle detector that is able to achieve 77% vehicle detection accuracy on UA-DETRAC dataset. The second observation is emergency vehicle have distinct siren sound that is detected using a novel acoustic detection algorithm on an edge device. The proposed system is able to detect emergency vehicles with an average accuracy of 99.4%.

Gene expression can be used to subtype breast cancer with improved prediction of risk of recurrence and treatment responsiveness over that obtained using routine immunohistochemistry (IHC). However, in the clinic, molecular profiling is primarily used for ER+ breast cancer, which is costly, tissue destructive, requires specialised platforms, and takes several weeks to obtain a result. Deep learning algorithms can effectively extract morphological patterns in digital histopathology images to predict molecular phenotypes quickly and cost-effectively. We propose a new, computationally efficient approach called hist2RNA inspired by bulk RNA sequencing techniques to predict the expression of 138 genes (incorporated from 6 commercially available molecular profiling tests), including luminal PAM50 subtype, from hematoxylin and eosin (H&E)-stained whole slide images (WSIs). The training phase involves the aggregation of extracted features for each patient from a pretrained model to predict gene expression at the patient level using annotated H&E images from The Cancer Genome Atlas (TCGA, n = 335). We demonstrate successful gene prediction on a held-out test set (n = 160, corr = 0.82 across patients, corr = 0.29 across genes) and perform exploratory analysis on an external tissue microarray (TMA) dataset (n = 498) with known IHC and survival information. Our model is able to predict gene expression and luminal PAM50 subtype (Luminal A versus Luminal B) on the TMA dataset with prognostic significance for overall survival in univariate analysis (c-index = 0.56, hazard ratio = 2.16 (95% CI 1.12–3.06), p < 5 × 10−3), and independent significance in multivariate analysis incorporating standard clinicopathological variables (c-index = 0.65, hazard ratio = 1.87 (95% CI 1.30–2.68), p < 5 × 10−3). The proposed strategy achieves superior performance while requiring less training time, resulting in less energy consumption and computational cost compared to patch-based models. Additiona

Over the last few decades, swarm intelligence (SI) has shown significant benefits in many practical applications. Real-world applications of swarm intelligence include disaster response and wildlife conservation. Swarm robots can collaborate to search for survivors, locate victims, and assess damage in hazardous environments during an earthquake or natural disaster. They can coordinate their movements and share data in real-time to increase their efficiency and effectiveness while guiding the survivors. In addition to tracking animal movements and behaviour, robots can guide animals to or away from specific areas. Sheep herding is a significant source of income in Australia that could be significantly enhanced if the human shepherd could be supported by single or multiple robots. Although the shepherding framework has become a popular SI mechanism, where a leading agent (sheepdog) controls a swarm of agents (sheep) to complete a task, controlling a swarm of agents is still not a trivial task, especially in the presence of some practical constraints. For example, most of the existing shepherding literature assumes that each swarm member has an unlimited sensing range to recognise all other members’ locations. However, this is not practical for physical systems. In addition, current approaches do not consider shepherding as a distributed system where an agent, namely a central unit, may observe the environment and commu- nicate with the shepherd to guide the swarm. However, this brings another hurdle when noisy communication channels between the central unit and the shepherd af- fect the success of the mission. Also, the literature lacks shepherding models that can cope with dynamic communication systems. Therefore, this thesis aims to design a multi-agent learning system for effective shepherding control systems in a partially observable environment under communication constraints. To achieve this goal, the thesis first introduces a new methodology to guide agents wh

Determining the best model for the neural network architecture and how to optimize the architecture with the metaheuristic Protis Approach is a subject of the study. A comprehensive investigation and utilization of metaheuristic methods are necessary. These methods aim to solve problems and adapt from the lifestyle of the amoeba protis. In this study, the proposed method modifies the life cycle of the amoeba, which consists of four phases: prophase, metaphase, anaphase, and telophase. These four phases are modified in the neural network architecture to optimize the appropriate number of hidden layers and produce an efficient architecture model. The results show that the protis approach optimized the neural network architecture, especially in generating hidden layers to improve the neural network model. Distinctive features of the results obtained are that the average range of degenerate neurons in the hidden layer is 0 to 35 neurons in each layer. The standard number of neurons makes it possible to solve the problem of determining the best model on the neural network architecture. The protis algorithm embedded in the protis recurrent neural network for categorical data measurements produces an average RMSE value, representing the difference between actual measurements and predictions, equal to 0.066. Consequently, the developed model surpasses the current classical neural network model in terms of performance. Regarding accuracy, the protis algorithm embedded in the neural network for categorical and time series data achieves an average precision of 0.952 and a recall of 0.950. The protis convolutional neural network achieves an accuracy of 95.9 %. Therefore, from the three tested datasets, the protis convolutional neural network exhibits the highest accuracy value

This paper examines the efficiency of a downlink non-orthogonal multiple access (NOMA) system by using a deep learning (DL)-based stacked long short-term memory (S-LSTM) scheme. The vehicle-to-vehicle (V2V) channel is considered to be time-selective as a result of node mobility and the presence of imprecise channel state information (CSI). The use of the fifth generation (5G) tapped delay line type C (TDL-C) independent and identically distributed (IID) fading channel models allows for the production of channel taps that properly replicate the Nakagami-m fading wireless channel. The paper examines the outage probability (OP) and symbol error rate (SER) of both traditional and suggested channel estimators. It analyzes these metrics under various fading parameters, pilot symbols (PS), learning rate (LR), and batch size. The training of deep neural network (DNN) models is performed using the Adam optimizer. Enhancing the signal-to-noise ratio (SNR) may decrease the SER which results in the enhanced identification of the downlink channel in NOMA cell-based systems. Reducing the LR has a positive effect on the SER, validating the analytical findings that indicate greater changes in DNN weights and larger validation mistakes when the LR is raised. Nevertheless, this benefit is accompanied by the drawback of more frequent updates, resulting in a delay in the model’s convergence.

To determine the optimal design of the wave energy converter (WEC) that can withstand extreme waveconditions, the short- and long-term extreme responses of the system need to be determined. This paper focuses on the extreme peak force distribution of the mooring force for a 1:30 scaled point absorber WEC. The basis of this analysis is the mooring force response obtained from a WEC-Sim model calibrated by wave tank experimental data. The extreme sea states have been chosen from a50-year environmental contour. Here, first, the long-term extreme response using the full sea state approach is obtained for three constant damping cases of the power take-off (PTO) system. Then, using a contour approach, the expected value of the extreme peak line (mooring) force distribution is computed for the sea states along an environmental contour. Further, for the most extremesea state, the extreme peak line force distribution is also computed where a survivability control system, based on a deep neural network (DNN), changes the PTO damping to minimize the peak mooring force in each zero up-crossing episode of surface elevation. The results show that in the absence of a control system, the zero PTO damping case is a conservative choice in the analysis of the long-term response and the design load. For the most extreme sea state along the environmental contour, the survivability control system slightly reduces the expected value of the extreme peak force distribution when compared with lower constant PTO damping configurations.

Background: Atrial fibrillation (AF) is one of the most common cardiac arrhythmias that affects millions of people each year worldwide and it is closely linked to increased risk of cardiovas-cular diseases such as stroke and heart failure. Machine learning methods have shown promising results in evaluating the risk of developing atrial fibrillation from the electrocardiogram. We aim to develop and evaluate one such algorithm on a large CODE dataset collected in Brazil. Methods: We used the CODE cohort to develop and test a model for AF risk prediction for individual patients from the raw ECG recordings without the use of additional digital biomarkers. The cohort is a collection of ECG recordings and annotations by the Telehealth Network of Minas Gerais, in Brazil. A convolutional neural network based on a residual network architecture was implemented to produce class probabilities for the classification of AF. The probabilities were used to develop a Cox proportional hazards model and a Kaplan-Meier model to carry out survival analysis. Hence, our model is able to perform risk prediction for the development of AF in patients without the condition. Results: The deep neural network model identified patients without indication of AF in the presented ECG but who will develop AF in the future with an AUC score of 0.845. From our survival model, we obtain that patients in the high-risk group (i.e. with the probability of a future AF case being >0.7) are 50% more likely to develop AF within 40 weeks, while patients belonging to the minimal-risk group (i.e. with the probability of a future AF case being less than or equal to 0.1) have >85% chance of remaining AF free up until after seven years. Conclusion: We developed and validated a model for AF risk prediction. If applied in clinical practice, the model possesses the potential of providing valuable and useful information in decision- making and patient management processes.

In the mmWave communication network, exhaustive beam search (EBS) assisted beam training is suggested for accurate beam alignment between the access point (AP) and the user equipment (UE) in the existing 3GPP standard. Nevertheless, the EBS method suffers from unacceptable training overheads, which may degrade the network throughput, especially when the beam space is large. In this paper, we propose an intelligent multi-peak beam training algorithm to tackle this issue. The main idea of the algorithm lies in, by applying binary encoding into the narrow beam index, several multi-peak wide beams can be constructed. Then, a deep neural network (DNN) is carefully designed and trained to decode the probed signal-to-noise ratios (SNRs) of multi-peak wide beams into the optimal narrow beam index. With the well-trained DNN, the optimal narrow beam can be identified online by feeding the probed SNRs of the multi-peak wide beams into the DNN. Simulation results show that the proposed algorithm outperforms the state of the arts in terms of overheads and throughput., Part of proceedings ISBN 9798350324662QC 20240307

We propose a deep neural network (DNN) based method that provides a posterior distribution of mean-opinion-score (MOS) for an input speech signal. The DNN outputs parameters of the posterior, mainly the posterior's mean and variance. The proposed method is referred to as deep posterior MOS (DeePMOS). The relevant training data is inherently limited in size (limited number of labeled samples) and noisy due to the subjective nature of human listeners. For robust training of DeePMOS, we use a combination of maximum-likelihood learning, stochastic gradient noise, and a student-teacher learning setup. Using the mean of the posterior as a point estimate, we evaluate standard performance measures of the proposed DeePMOS. The results show comparable performance with existing DNN-based methods that only provide point estimates of the MOS. Then we provide an ablation study showing the importance of various components in DeePMOS., QC 20231010

The increasing occurrence of structural deformations in the electrodes of the jelly roll has raised quality concerns during battery manufacturing, emphasizing the need to detect them automatically with the advanced techniques. This thesis aims to explore and provide two models based on traditional computer vision (CV) and deep neural network (DNN) techniques using computed tomography (CT) scan images of jelly rolls to ensure that the product is of high quality. For both approaches, electrode peaks as keypoints of anodes and cathodes in prismatic lithium battery jelly rolls are detected to extract the geometric features to identify if a particular jelly roll has some structural deformations. For traditional CV methods, the images undergo some pre-processing steps, extraction of foreground through adaptive thresholding, and morphological operations to extract contour edges, followed by applying Harris corner detector to detect electrode peaks. However, this approach shows limitations in detecting small or negative distance differences in deformed images. Furthermore, this study proposes another approach based on supervised transfer learning using pre-trained deep learning models on annotated data. After exploring different architectures, the VGG19 model pre-trained on ImageNet dataset outperformed as compared to other architectures, even with insufficient training data, achieving a maximum accuracy of 93.13 % for 1-pixel distance, 98.87 % for 5-pixel distance and 99.29 % for 10-pixel distance on test data, where the performance metrics, such as Percentage of Correct Keypoint (PCK), Mean-Square Error and Huber loss are utilized. As a result, this baseline proves to be a valuable tool for detecting structural deformations in jelly rolls. Moreover, a GUI-based executable application is developed using both approaches for raising the OK or NG flags for detecting structural deformations in each jelly roll., Den ökande förekomsten av strukturella deformationer av elektroderna i så kallade jelly rolls har väckt kvalitetsproblem under batteritillverkning, och betonat behovet av att upptäcka dem automatiskt med avancerade tekniker. Denna avhandling syftar till att utforska och tillhandahålla två modeller baserade på traditionell datorseende (CV) och djupa neurala nätverk (DNN) tekniker med hjälp av bilder från datortomografisk skanning (CT) av jelly rolls för att säkerställa att produkten är av hög kvalitet. För båda metoderna detekteras elektrodtoppar som nyckelpunkter på anoder och katoder i prismatiska litiumbatteriers jelly rolls för att extrahera de geometriska egenskaperna för att identifiera om en viss jelly roll har några strukturella deformationer. För traditionella CV-metoder genomgår bilderna några förbehandlingssteg, extraktion av förgrund genom adaptiv tröskling och morfologiska operationer för att extrahera konturkanter, följt av användning av Harris hörndetektor för att upptäcka elektrodtoppar. Denna metod visar dock begränsningar i att detektera små eller negativa avståndsskillnader i deformerade bilder. Vidare föreslår denna studie en annan metod baserad på övervakad överföringsinlärning med förtränade djupinlärningsmodeller på annoterade data. Efter att ha utforskat olika arkitekturer presterade VGG19-modellen förtränad på ImageNet-datasetet bättre jämfört med andra arkitekturer, även med otillräcklig träningsdata, och uppnådde en maximal noggrannhet på 91,56% för 1-pixels avstånd, 97,49% för 5-pixels avstånd och 98,91% för 10-pixels avstånd på testdata, där prestationsmått som procentandel av korrekta nyckelpunkter (PCK), medelkvadratfel och Huber-förlust används. Som ett resultat visar sig denna grundlinje vara ett värdefullt verktyg för att upptäcka strukturella deformationer i jelly rolls. Dessutom har exekverbar applikation med grafiskt gränssnitt utvecklats med båda metoderna för att höja OK/NG-flaggorna för att upptäcka strukturella deformationer i

Variable Renewable Energy (VRE) sources are characterized by production uncertainty that is largely due to weather forecast errors. This leads to greater volumes in the real-time balancing markets that drive Imbalance Market (IM) price higher, creating economic opportunities for flexibility providers. However, lack of information on foreseen IM prices and regulation states at the time of day-ahead market closure, reduces the flexibility providers' opportunity of optimizing their portfolio. The objective of this paper is to investigate to which extent there is a correlation between the IM prices and weather parameters in the Dutch Imbalance Market. A Deep Learning (DL) model based on Feed-Forward Deep Neural Network (FFDNN) is developed with the aim to support VRE asset owners and flexibility providers to predict IM price ranges and regulation states in the day-ahead horizon. The parameters considered are temperature, solar radiation, wind speed, relative humidity, and cloud cover. A benchmark model using Support Vector Machine (SVM) is used to compare with the DL's model performance. Both models are trained and tested using data from the weather prediction model provided by Royal Netherlands Meteorological Institute (KNMI), and historical IM prices from the Dutch Transmission System Operator (Tenne'T), for the years 2018–2020.

Proton exchange membrane fuel cells' (PEMFCs') degradation is a main problem that must be solved for their commercialization. In this thesis, we employ data-driven prognostic models to forecast PEMFC voltage degradation utilizing artificial neural networks (ANN), deep neural networks (DNN), and long short-term memory (LSTM) techniques. The suggested models are developed and tested using experimental information obtained through PEMFC stack testing, and their performance is assessed using a variety of metrics, such as root mean square error (RMSE). The findings show that the DNN technique performs better than the other methods, indicating that the model can reliably and precisely forecast the PEMFC stack voltage decline. In addition, we examined the models' performance in terms of the neural network designs' number of neurons and layers. The results demonstrate a trade-off between prediction accuracy and computational complexity, with an increase in neurons and layers able to enhance prediction accuracy. In conclusion, the suggested data-driven prognostic models may offer precise and trustworthy forecasts of PEMFC stack voltage degradation, which can aid in optimizing system design and operation. In order to improve the precision and effectiveness of the prediction process, future research can concentrate on investigating the applicability of these models to specific degradation modes, such as catalyst and membrane degradation, and developing more sophisticated data-driven prognostic models. Moreover, combining data-driven and physics-based models can offer a thorough strategy for PEMFC degradation prognostics.Keywords: Proton exchange membrane fuel cell, Degradation, Prognostics, Artificial neural network, Deep neural network, Long short-term memory, Data-driven model.

Data-driven prognostics and health management is an important part of the future industry. It allows the detection of system faults and estimation of its remaining useful life (RUL) to anticipate failures and schedule appropriate prescriptive maintenance actions. Moreover, with the development of data acquisition tools in such industries, it is possible to collect large amount of data from similar systems, which prompts the use of machine learning algorithms for efficient estimation of the system RUL. However, due to different type of faults and degradation rates that can occur in real processes, there exists high variability of the end-of-life (¿¿¿) time of each system degradation trajectory, making more difficult to fix a failure threshold and consequently generate several uncertainties in the estimation of RUL. To address this situation, this paper proposes a new data-driven approach for estimating the system RUL when dealing with the variability of degradation trends and unknown failure thresholds. Particularly, the proposed approach combines two RUL techniques, recursive and direct RUL estimation. First, the historical collected raw data are fed into a processing algorithm to construct prognostic health indicators (HIs) and choose the one that characterizes well the system’s degradation trajectory. This latter indicator allows identifying the high and low ¿¿¿ amplitude values from the historical data and is used to build a recursive prediction model that estimates in long-term the degradation trend evolution. After that, the trained model forecasts every degradation trend which ¿¿¿ amplitude is less than the predefined high value. Thus, a set of possible RULs of each trajectory can be predicted. Finally, the ensemble of the derived RULs and their HI trajectories are fused to directly estimate the final RUL. The proposed approach is applied to a subway door system with multiple degradation scenarios while taking into account different operating conditions., The project has been 65% cofinanced by the European Regional Development Fund (ERDF) through the Interreg V-A Spain France Andorra programme (POCTEFA 2014–2020). POCTEFA aims to reinforce the economic and social integration of the French-Spanish-Andorran border. Its support is focused on developing economic, social and environmental cross-border activities through joint strategies favoring sustainable territorial development. Also, this work was supported by the ANR DATAZERO (contract ‘‘ANR-15-CE25-0012’’) project and by the EIPHI Graduate school (contract "ANR-19-CE25-0016")., Postprint (published version)

Sound source separation is a popular and active research area, especially with modern machine learning techniques. In this thesis, the focus is on single-channel separation of two speakers into individual streams, and specifically considering the case where two speakers are also accompanied by background noise. There are different methods to separate speakers and in this thesis three different methods are evaluated: the Conv-TasNet, the DPTNet, and the FaSNetTAC. The methods were used to train models to perform the sound source separation. These models were evaluated and validated through three experiments. Firstly, previous results for the chosen separation methods were reproduced. Secondly, appropriate models applicable for NFC's datasets and applications were created, to fulfill the aim of this thesis. Lastly, all models were evaluated on an independent dataset, similar to datasets from NFC. The results were evaluated using the metrics SI-SNRi and SDRi. This thesis provides recommended models and methods suitable for NFC applications, especially concluding that the Conv-TasNet and the DPTNet are reasonable choices.

Seismic activity prediction has been a challenging research domain: in this regard, accurate prediction using historical data is an intricate task. Numerous machine learning and traditional approaches have been presented lately for seismic activity prediction; however, no generalizable model exists. In this work, we consider seismic activity predication as a binary classification problem, and propose a deep neural network architecture for the classification problem, using historical data from Chile, Hindukush, and Southern California. After obtaining the data for the three regions, a data cleaning process was used, which was followed by a feature engineering step, to create multiple new features based on various seismic laws. Afterwards, the proposed model was trained on the data, for improved prediction of the seismic activity. The performance of the proposed model was evaluated and compared with extant techniques, such as random forest, support vector machine, and logistic regression. The proposed model achieved accuracy scores of 98.28%, 95.13%, and 99.29% on the Chile, Hindukush, and Southern California datasets, respectively, which were higher than the current benchmark model and classifiers. In addition, we also conducted out-sample testing, where the evaluation metrics confirmed the generality of our proposed approach.

In this paper, the local correspondence between synthetic aperture radar (SAR) images and optical images is proposed using an image feature-based keypoint-matching algorithm. To achieve accurate matching, common image features were obtained at the corresponding locations. Since the appearance of SAR and optical images is different, it was difficult to find similar features to account for geometric corrections. In this work, an image translator, which was built with a DNN (deep neural network) and trained by conditional generative adversarial networks (cGANs) with edge enhancement, was employed to find the corresponding locations between SAR and optical images. When using conventional cGANs, many blurs appear in the translated images and they degrade keypoint-matching accuracy. Therefore, a novel method applying an edge enhancement filter in the cGANs structure was proposed to find the corresponding points between SAR and optical images to accurately register images from different sensors. The results suggested that the proposed method could accurately estimate the corresponding points between SAR and optical images.

Imbalanced Data (ID) is a problem that deters Machine Learning (ML) models from achieving satisfactory results. ID is the occurrence of a situation where the quantity of the samples belonging to one class outnumbers that of the other by a wide margin, making such models’ learning process biased towards the majority class. In recent years, to address this issue, several solutions have been put forward, which opt for either synthetically generating new data for the minority class or reducing the number of majority classes to balance the data. Hence, in this paper, we investigate the effectiveness of methods based on Deep Neural Networks (DNNs) and Convolutional Neural Networks (CNNs) mixed with a variety of well-known imbalanced data solutions meaning oversampling and undersampling. Then, we propose a CNN-based model in combination with SMOTE to effectively handle imbalanced data. To evaluate our methods, we have used KEEL, breast cancer, and Z-Alizadeh Sani datasets. In order to achieve reliable results, we conducted our experiments 100 times with randomly shuffled data distributions. The classification results demonstrate that the mixed Synthetic Minority Oversampling Technique (SMOTE)-Normalization-CNN outperforms different methodologies achieving 99.08% accuracy on the 24 imbalanced datasets. Therefore, the proposed mixed model can be applied to imbalanced binary classification problems on other real datasets., Validerad;2023;Nivå 2;2023-03-22 (joosat);Licens fulltext: CC BY License

Terahertz (THz) spectrum is identified as a potential enabler for advanced sensing and positioning, where THz-Time domain spectroscopy (THz-TDS) is specified for investigating the unique material properties. The transmission THz-TDS measures the light absorption of materials. This paper proposes a novel low-complex deep neural network (DNN)-based multi-class classification architecture to sense a wide variety of materials from the transmission spectroscopy. Based on the spectroscopic measurements made across a chosen THz region of interest, DNN extracts and learns the distinctive crystal structure of materials as features. With sufficient quantities of noisy spectroscopic data and labels, we train and validate the model. In low SNR regions, the proposed DNN classification architecture achieves about 92% success rate, which is greater than those of the state-of-the-art methods.