Search

Pattern recognition systems always misclassify anomalies, which can be dangerous for uninformed users. Therefore, anomalies must be filtered out from each classification. The main challenge for the anomaly filter design is the huge number of possible anomaly samples compared with the number of samples in the training set. Tailoring the filter for the given classifier is just the first step in this reduction. Paper tests the hypothesis that the filter trained in avoiding "near" anomalies will also refuse the "far" anomalies, and the anomaly detector is then just a classifier distinguishing between "far real" and "near anomaly" samples. As a "far real" samples generator was used, a Generative Adversarial Network (GAN) fake generator that transforms normally distributed random seeds into fakes similar to the training samples. The paper proves the assumption that seeds unused in fake training will generate anomalies. These seeds are distinguished according to their Chebyshev norms. While the fakes have seeds within the hypersphere with a given radius, the near anomalies have seeds within the sphere near cover. Experiments with various anomaly test sets have shown that GAN-based anomaly detectors create a reliable anti-anomaly shield using the abovementioned assumptions. The proposed anomaly detector is tailored to the given classifier, but its limitation is due to the need for the availability of the database on which the classifier was trained. [ABSTRACT FROM AUTHOR]

Lithium-ion batteries are widely used in electric vehicles, energy storage power stations, and many other applications. Accurate and reliable monitoring of battery health status and remaining capacity is the key to establish a lithium-ion cell management system. In this paper, based on a Bayesian optimization algorithm, a deep neural network is structured to evaluate the whole charging curve of the battery using partial charging curve data as input. A 0.74 Ah battery is used for experiments, and the effect of different input data lengths is also investigated to check the high flexibility of the approach. The consequences show that using only 20 points of partial charging data as input, the whole charging profile of a cell can be exactly predicted with a root-mean-square error (RMSE) of less than 19.16 mAh (2.59% of the nominal capacity of 0.74 Ah), and its mean absolute percentage error (MAPE) is less than 1.84%. In addition, critical information including battery state-of-charge (SOC) and state-of-health (SOH) can be extracted in this way to provide a basis for safe and long-lasting battery operation. [ABSTRACT FROM AUTHOR]

The application of deep neural network has achieved remarkable success in object detection. However, the network structures should be still evolved consistently and tuned finely to acquire better performance. This gears to the continuous demands on high performance in those complex scenes, where multi-scale objects to be detected are located here and there. To this end, this paper proposes a network structure called Multi-Scale Coupled Attention (MSCA) under the framework of self-attention learning with methodologies of importance assessment. Architecturally, it consists of a Multi-Scale Coupled Channel Attention (MSCCA) module, and a Multi-Scale Coupled Spatial Attention (MSCSA) module. Specifically, the MSCCA module is developed to achieve the goal of self-attention learning linearly on the multi-scale channels. In parallel, the MSCSA module is constructed to achieve this goal nonlinearly on the multi-scale spatial grids. The MSCCA and MSSCA modules can be connected together into a sequence, which can be used as a plugin to develop end-to-end learning models for object detection. Finally, our proposed network is compared on two public datasets with 13 classical or state-of-the-art models, including the Faster R-CNN, Cascade R-CNN, RetinaNet, SSD, PP-YOLO, YOLO v3, YOLO v5, YOLO v7, YOLOX, DETR, conditional DETR, UP-DETR and FP-DETR. Comparative experimental results with numerical scores, the ablation study, and the performance behaviour all demonstrate the effectiveness of our proposed model. [ABSTRACT FROM AUTHOR]

In this paper, we study the post-hoc calibration of modern neural networks, a problem that has drawn a lot of attention in recent years. Despite the plethora of calibration methods proposed, there is no consensus yet on the inherent complexity of the task and, while some authors claim that simple functions solve the problem, others suggest that more expressive models are needed to capture misscalibration. As a first approach, we focus on the task of confidence scaling, specifically on post-hoc methods that generalize Temperature Scaling, which we refer to as the Adaptive Temperature Scaling family. We begin by demonstrating that while complex models like neural networks provide an advantage when there is ample data, they fail in scenarios where it is limited, notably common in fields like medical diagnosis. We then show how under this ideal data conditions the more expressive methods learn a relationship between the entropy of a prediction and its level of overconfidence, and based on this observation, we propose Entropy-based Temperature Scaling, a simple method that scales the confidence of a prediction according to this relationship. Results show that our method obtains state-of-the-art performance and is robust against data scarcity. Moreover, our proposed model enables a deeper understanding of the calibration process by the interpretation of the entropy as a measure of uncertainty in the network outputs. [ABSTRACT FROM AUTHOR]

Accurately predicting customer churn is considered crucial by businesses in order to take proactive measures to retain their customers and avoid financial losses. In this paper, a customer churn prediction model is proposed that incorporates deep neural networks and imbalanced data techniques. The approach involves applying oversampling and undersampling methods to address class imbalances in the dataset. The model's performance is evaluated using various evaluation metrics and compared to other methods. The results demonstrate that superior performance in predicting customer churn is achieved by the proposed model compared to traditional statistical methods and deep neural networks without imbalanced data techniques. Important implications for businesses seeking to reduce customer churn and improve customer retention are provided by our findings. By using the suggested model, customers can be retained, and financial losses can be avoided proactively. [ABSTRACT FROM AUTHOR]

Controlled de-orbiting is crucial for a low earth orbiting (LEO) satellite or capsule intending to land in a desired location and to prevent damage to people and property on the ground caused by debris. Drag modulation is one possible mechanism to control de-orbiting, exploiting atmospheric drag variation to reduce the necessary orbital energy from the initial conditions to the re-entry interface. In this context, this paper proposes a novel targeted de-orbiting Artificial Neural Network (ANN) and drag-based guidance algorithm for a LEO artificial satellite. It relies on the combination of empirical and analytical relations with Deep Neural Networks (DNNs) to estimate the main parameters of the optimal control law in real time. The training set is composed of several optimal control solutions generated with a previously developed optimal control algorithm, exploiting the formulation in equinoctial modified orbital parameters to manage the large time scale of the problem and the computational cost. An innovative procedure for the training set generation led to the achievement of general results with a limited number of samples. The successful outcome of the guidance algorithm on over 1000 cases demonstrates its robustness and generalization of results. To conclude, a Linear Quadratic Regulator (LQR) feedback control is applied to one case to deal with a more realistic and uncertain density model. • Analytical optimal guidance for a satellite de-orbiting from Low Earth Orbit. • A novel approach based on Deep Neural Networks. • Prediction of the key parameters of the optimal control law in real time. • A Monte Carlo analysis showed the algorithm's robustness and generalization. • Successful feedback control thanks to a Linear Quadratic Regulator (LQR). [ABSTRACT FROM AUTHOR]

Deformable linear objects (DLOs), such as rods, cables, and ropes, play important roles in daily life. However, manipulation of DLOs is challenging as large geometrically nonlinear deformations may occur during the manipulation process. This problem is made even more difficult as the different deformation modes (e.g., stretching, bending, and twisting) may result in elastic instabilities during manipulation. In this paper, we formulate a physics-guided data-driven method to solve a challenging manipulation task—accurately deploying a DLO (an elastic rod) onto a rigid substrate along various prescribed patterns. Our framework combines machine learning, scaling analysis, and physical simulations to develop a physics-based neural controller for deployment. We explore the complex interplay between the gravitational and elastic energies of the manipulated DLO and obtain a control method for DLO deployment that is robust against friction and material properties. Out of the numerous geometrical and material properties of the rod and substrate, we show that only three non-dimensional parameters are needed to describe the deployment process with physical analysis. Therefore, the essence of the controlling law for the manipulation task can be constructed with a low-dimensional model, drastically increasing the computation speed. The effectiveness of our optimal control scheme is shown through a comprehensive robotic case study comparing against a heuristic control method for deploying rods for a wide variety of patterns. In addition to this, we also showcase the practicality of our control scheme by having a robot accomplish challenging high-level tasks such as mimicking human handwriting, cable placement, and tying knots. [ABSTRACT FROM AUTHOR]

The object of the research is the line segmentation task. To recognize the handwritten text from the documents in image format offline handwriting recognition technology is used. The text recognizer module accepts input as separate lines, so one of the important preprocessing steps is the detection and splitting of all handwritten text into distinct lines. In this paper, the handwritten text line segmentation task, its requirements, problems, and challenges are examined. Two main approaches for this task that are used in modern recognition systems are reviewed. These approaches are statistical projection-based methods and neural-based methods. Multiple works and research papers for each type of approach are reviewed analyzing their strengths and weaknesses considering the described tasks, constraints, and input data peculiarities. Overall acquired results are formed in a single table for comparison. Based on the latest works that utilize deep neural networks the new possibilities of using these methods in recognition systems are described that were unavailable with traditional statistical segmentation approaches. The constructive conclusions are made based on the review, describing the main pros and cons of these two approaches for the line segmentation task. These results can be further used for the correct selection of suitable methods in handwriting recognition systems to improve their performance and quality, and for further research in this area. [ABSTRACT FROM AUTHOR]

Shadow removal for document images is an essential task for digitized document applications. Recent shadow removal models have been trained on pairs of shadow images and shadow-free images. However, obtaining a large, diverse dataset for document shadow removal takes time and effort. Thus, only small real datasets are available. Graphic renderers have been used to synthesize shadows to create relatively large datasets. However, the limited number of unique documents and the limited lighting environments adversely affect the network performance. This paper presents a large-scale, diverse dataset called the Synthetic Document with Diverse Shadows (SynDocDS) dataset. The SynDocDS comprises rendered images with diverse shadows augmented by a physics-based illumination model, which can be utilized to obtain a more robust and high-performance deep shadow removal network. In this paper, we further propose a Dual Shadow Fusion Network (DSFN). Unlike natural images, document images often have constant background colors requiring a high understanding of global color features for training a deep shadow removal network. The DSFN has a high global color comprehension and understanding of shadow regions and merges shadow attentions and features efficiently. We conduct experiments on three publicly available datasets, the OSR, Kligler's, and Jung's datasets, to validate our proposed method's effectiveness. In comparison to training on existing synthetic datasets, our model training on the SynDocDS dataset achieves an enhancement in the PSNR and SSIM, increasing them from 23.00 dB to 25.70 dB and 0.959 to 0.971 on average. In addition, the experiments demonstrated that our DSFN clearly outperformed other networks across multiple metrics, including the PSNR, the SSIM, and its impact on OCR performance. [ABSTRACT FROM AUTHOR]

Flettner rotors are wind propulsion systems using the Magnus effect to generate thrust, thereby reduce fuel consumption and carbon emissions in the ships. However, rotor unbalance can cause excessive vibrations and energy loss, affecting the performance and stability of the system. There is a need to have a system onboard, which can predict the vibrations. The paper proposes a deep learning approach to predict the vibrations and unbalanced forces of a Flettner rotor based on the data of ECO Flettner rotor onboard the vessel MV Fehn pollux. The paper develops two methods to estimate the direction and magnitude of the unbalanced forces using the reading values of the strain gauges. The work also compares two recurrent neural network models, namely Long-short term memory and Gated Recurrent Unit, for vibration prediction and evaluates their performance using Mean Absolute Error and Root Mean Squared Error metrics. The results show that Longshort term memory model outperforms Gated Recurrent Unit model in prediction accuracy and can be implemented on the system onboard to monitor and prevent rotor unbalance. The paper also suggests some possible solutions for automatic self-balancing of the rotor and identifies some areas for future work. [ABSTRACT FROM AUTHOR]

Remote sensing scene classification is the basis of advanced smart urban planning tasks such as urban functional zone division and land use type identification. In recent years, a wide range of emerging data sources is receiving progressive attention for urban features extraction, such as satellites, unmanned aerial vehicles (UAVs), and ground sensors. How to effectively utilize these multi-view data jointly to improve scene classification performance has become a hot topic of remote sensing challenge. Existing feature fusion methods tend to map data from different views into a common feature space, which is often difficult to find when the data between views differ greatly. Furthermore, because these methods require data from all views as input, they are not flexible enough to handle situations where there is only one view input when inferring. To address the aforementioned issues, a novel Coupled Parallel Architecture (CPA) using Weighted Collaboration Fusion Constrained by Consistency Between Views (CBV-WCF) is proposed in this paper. In the training phase, the CBV module reduces the impact of the heterogeneous gap across views by capturing the consistency information between views. Well the WCF module is used to fully mine and effectively fuse the complementary information between views to improve the performance of downstream tasks. In the inference phase, the proposed architecture can effectively improve the classification performance in both cases of multi-view and single-view input. Our method is evaluated on air-ground dual-view scene classification, which is a typical multi-view task with large image differences between views. Experimental results on two publicly available air-ground dual-view datasets demonstrate that the proposed framework significantly improves classification performance while bringing some inspiration and new solutions to multi-view tasks. The code of this paper will be published at: . [ABSTRACT FROM AUTHOR]

Background: The discovery of anticancer drug combinations is a crucial work of anticancer treatment. In recent years, pre-screening drug combinations with synergistic effects in a large-scale search space adopting computational methods, especially deep learning methods, is increasingly popular with researchers. Although achievements have been made to predict anticancer synergistic drug combinations based on deep learning, the application of multi-task learning in this field is relatively rare. The successful practice of multi-task learning in various fields shows that it can effectively learn multiple tasks jointly and improve the performance of all the tasks. Methods: In this paper, we propose MTLSynergy which is based on multi-task learning and deep neural networks to predict synergistic anticancer drug combinations. It simultaneously learns two crucial prediction tasks in anticancer treatment, which are synergy prediction of drug combinations and sensitivity prediction of monotherapy. And MTLSynergy integrates the classification and regression of prediction tasks into the same model. Moreover, autoencoders are employed to reduce the dimensions of input features. Results: Compared with the previous methods listed in this paper, MTLSynergy achieves the lowest mean square error of 216.47 and the highest Pearson correlation coefficient of 0.76 on the drug synergy prediction task. On the corresponding classification task, the area under the receiver operator characteristics curve and the area under the precision–recall curve are 0.90 and 0.62, respectively, which are equivalent to the comparison methods. Through the ablation study, we verify that multi-task learning and autoencoder both have a positive effect on prediction performance. In addition, the prediction results of MTLSynergy in many cases are also consistent with previous studies. Conclusion: Our study suggests that multi-task learning is significantly beneficial for both drug synergy prediction and monotherapy sensitivity prediction when combining these two tasks into one model. The ability of MTLSynergy to discover new anticancer synergistic drug combinations noteworthily outperforms other state-of-the-art methods. MTLSynergy promises to be a powerful tool to pre-screen anticancer synergistic drug combinations. [ABSTRACT FROM AUTHOR]

To achieve successful prevention of fire incidents originating from human activities, it is imperative to possess a thorough understanding. This paper introduces a machine learning approach, specifically utilizing deep neural networks (DNN), to develop predictive models for fire occurrence in Keelung City, Taiwan. It investigates ten factors across demographic, architectural, and economic domains through spatial analysis and thematic maps generated from geographic information system data. These factors are then integrated as inputs for the DNN model. Through 50 iterations, performance indices including the coefficient of determination (R2), root mean square error (RMSE), variance accounted for (VAF), prediction interval (PI), mean absolute error (MAE), weighted index (WI), weighted mean absolute percentage error (WMAPE), Nash–Sutcliffe efficiency (NS), and the ratio of performance to deviation (RPD) are computed, with average values of 0.89, 7.30 × 10−2, 89.21, 1.63, 4.90 × 10−2, 0.97, 2.92 × 10−1, 0.88, and 4.84, respectively. The model's predictions, compared with historical data, demonstrate its efficacy. Additionally, this study explores the impact of various urban renewal strategies using the DNN model, highlighting the significant influence of economic factors on fire incidence. This underscores the importance of economic factors in mitigating fire incidents and emphasizes their consideration in urban renewal planning. [ABSTRACT FROM AUTHOR]

To improve convergence speed of deep neural network trained by gradient descent methods (GDMs), this paper proposed a novel fractional order gradient descent optimizer (FOAdam) based on Caputo operator and Adam algorithm, and analyzed the impact of fractional order on convergence performance using optimization theory and simulation. Furthermore, to address the trade-off between convergence speed and precision under fixed-order conditions, a fractional order scheduler (FOS) was designed based on the Connections Cloud Model (CCM). Numerical experiments conducted on MNIST and CIFAR-10 datasets showed that FOS-FOAdam outperformed other mainstream GDMs in terms of convergence speed and precision. Finally, FOS-FOAdam was applied in recognizing and classifying image datasets of cuttings and cores at the logging site, demonstrating good prospects for practical engineering applications. [ABSTRACT FROM AUTHOR]

The advanced development of communication technologies has made covert communications quite challenging to be recognized. By altering an entity's timing behavior during overt network communication, a covert timing channel (CTC) provides a way to leak sensitive data. This is becoming a serious threat due to the ability to transmit hidden messages without being detected by traditional security systems such as proxies and firewalls. In this paper, we explore the automatic classification and identification of covert timing channels using deep neural networks, namely, Long Short-Term Memory (LSTM), 1D-Convolutional Neural networks (1D-CNN), and a hybrid of LSTM and 1D-CNN. These classifiers have been trained and tested using sequence real inter-arrival times datasets labeled with covert or legitimate. The stream length differs for each dataset; (32, 64, 128, 256, and 512 bytes). Traditional machine-learning models have also been developed for comparisons and evaluation. The evaluation results showed that the hybrid of LSTM and CNN model outperforms other models either developed by deep neural networks or traditional machine learning with an accuracy of %97.5. [ABSTRACT FROM AUTHOR]

• A graph neural network is proposed for learning multi-satellite scheduling policy. • It analyzes the differential impact of various constraints in scheduling. • It models the multi-satellite scheduling problem as a heterogeneous graph. • It designs a variant transformer to learn various relations within graph. • It designs a two-stage model to learn a multi-satellite scheduling policy. In this paper, we investigate the multi-satellite scheduling (MSS) problem, a combinatorial optimization problem (COP) that aims to enhance satellite utilization. Numerous scholars have conducted in-depth studies on this matter. Handcrafted policies are commonly employed in various algorithms. Despite the use of graphical representations, designing policies manually becomes increasingly difficult, particularly as the number of satellites and tasks grows. Consequently, we propose a novel approach for automatically summarizing graph structures and learning policies for MSS problem. The proposed approach is developed based on graph neural network, Transformer, and sequence to sequence framework. First, MSS problem is modeled as a heterogeneous graph following various constraints. Next, the encoder, which is based on GNN and Transformer, is designed to summarize crucial information from the aforementioned graph. Finally, a two-stage decoder is proposed taking into account dynamic information in stepwise solution generation. Since obtaining optimal solutions in practical applications is challenging, we train the model using reinforcement learning. The results of the experiment demonstrate that our proposed method performs effectively in various scenarios. The quality of the solutions generated through this method transcends that of conventional construction methods. Additionally, the method generates solutions quickly, which are comparable in quality to those produced by meta-heuristic algorithms. [ABSTRACT FROM AUTHOR]

Deep neural networks excel in supervised learning tasks but are constrained by the need for extensive labeled data. Self-supervised learning emerges as a promising alternative, allowing models to learn without explicit labels. Information theory has shaped deep neural networks, particularly the information bottleneck principle. This principle optimizes the trade-off between compression and preserving relevant information, providing a foundation for efficient network design in supervised contexts. However, its precise role and adaptation in self-supervised learning remain unclear. In this work, we scrutinize various self-supervised learning approaches from an information-theoretic perspective, introducing a unified framework that encapsulates the self-supervised information-theoretic learning problem. This framework includes multiple encoders and decoders, suggesting that all existing work on self-supervised learning can be seen as specific instances. We aim to unify these approaches to understand their underlying principles better and address the main challenge: many works present different frameworks with differing theories that may seem contradictory. By weaving existing research into a cohesive narrative, we delve into contemporary self-supervised methodologies, spotlight potential research areas, and highlight inherent challenges. Moreover, we discuss how to estimate information-theoretic quantities and their associated empirical problems. Overall, this paper provides a comprehensive review of the intersection of information theory, self-supervised learning, and deep neural networks, aiming for a better understanding through our proposed unified approach. [ABSTRACT FROM AUTHOR]

Domain generalization (DG) techniques strive to attain the ability to generalize to an unfamiliar target domain solely based on training data originating from the source domains. Despite the increasing attention given to learning from multiple training domains through the application of various forms of invariance across those domains, the enhancements observed in comparison to ERM are nearly insignificant under specified evaluation rules. In this paper, we demonstrate that the disentanglement of spurious and invariant features is a challenging task in conventional training since ERM simply minimizes the loss and does not exploit invariance among domains. To address this issue, we introduce an effective method called specific domain training (SDT), which detects the spurious features and makes them more discernible. By exploiting a masking strategy and weight averaging, it decreases their harmful effects. We provide theoretical and experimental evidence to show the effectiveness of SDT for out-of-distribution generalization. Notably, SDT achieves comparable results to SWAD, the state of the art in DomainBed benchmarks. [ABSTRACT FROM AUTHOR]

Connections between circular RNAs (circRNAs) and microRNAs (miRNAs) assume a pivotal position in the onset, evolution, diagnosis and treatment of diseases and tumors. Selecting the most potential circRNA-related miRNAs and taking advantage of them as the biological markers or drug targets could be conducive to dealing with complex human diseases through preventive strategies, diagnostic procedures and therapeutic approaches. Compared to traditional biological experiments, leveraging computational models to integrate diverse biological data in order to infer potential associations proves to be a more efficient and cost-effective approach. This paper developed a model of Convolutional Autoencoder for CircRNA–MiRNA Associations (CA-CMA) prediction. Initially, this model merged the natural language characteristics of the circRNA and miRNA sequence with the features of circRNA–miRNA interactions. Subsequently, it utilized all circRNA–miRNA pairs to construct a molecular association network, which was then fine-tuned by labeled samples to optimize the network parameters. Finally, the prediction outcome is obtained by utilizing the deep neural networks classifier. This model innovatively combines the likelihood objective that preserves the neighborhood through optimization, to learn the continuous feature representation of words and preserve the spatial information of two-dimensional signals. During the process of 5-fold cross-validation, CA-CMA exhibited exceptional performance compared to numerous prior computational approaches, as evidenced by its mean area under the receiver operating characteristic curve of 0.9138 and a minimal SD of 0.0024. Furthermore, recent literature has confirmed the accuracy of 25 out of the top 30 circRNA–miRNA pairs identified with the highest CA-CMA scores during case studies. The results of these experiments highlight the robustness and versatility of our model. [ABSTRACT FROM AUTHOR]

LiDAR has high accuracy and resolution and is widely used in various fields. In particular, phase-modulated continuous-wave (PhMCW) LiDAR has merits such as low power, high precision, and no need for laser frequency modulation. However, with decreasing signal-to-noise ratio (SNR), the noise on the signal waveform becomes so severe that the current methods to extract the time-of-flight are no longer feasible. In this paper, a novel method that uses deep neural networks to measure the pulse width is proposed. The effects of distance resolution and SNR on the performance are explored. Recognition accuracy reaches 81.4% at a 0.1 m distance resolution and the SNR is as low as 2. We simulate a scene that contains a vehicle, a tree, a house, and a background located up to 6 m away. The reconstructed point cloud has good fidelity, the object contours are clear, and the features are restored. More precisely, the three distances are 4.73 cm, 6.00 cm, and 7.19 cm, respectively, showing that the performance of the proposed method is excellent. To the best of our knowledge, this is the first work that employs a neural network to directly process LiDAR signals and to extract their time-of-flight. [ABSTRACT FROM AUTHOR]

Recently, leveraging deep neural networks for automated colorectal polyp segmentation has emerged as a hot topic due to the favored advantages in evading the limitations of visual inspection, e.g., overwork and subjectivity. However, most existing methods do not pay enough attention to the uncertain areas of colonoscopy images and often provide unsatisfactory segmentation performance. In this paper, we propose a novel boundary uncertainty aware network (BUNet) for precise and robust colorectal polyp segmentation. Specifically, considering that polyps vary greatly in size and shape, we first adopt a pyramid vision transformer encoder to learn multi-scale feature representations. Then, a simple yet effective boundary exploration module (BEM) is proposed to explore boundary cues from the low-level features. To make the network focus on the ambiguous area where the prediction score is biased to neither the foreground nor the background, we further introduce a boundary uncertainty aware module (BUM) that explores error-prone regions from the high-level features with the assistance of boundary cues provided by the BEM. Through the top-down hybrid deep supervision, our BUNet implements coarse-to-fine polyp segmentation and finally localizes polyp regions precisely. Extensive experiments on five public datasets show that BUNet is superior to thirteen competing methods in terms of both effectiveness and generalization ability. • A boundary uncertainty aware network is proposed for accurate polyp segmentation. • A boundary exploration module is proposed to explore boundary cues of polyps. • A boundary uncertainty aware module is proposed to seek error-prone regions. • The proposed network achieves high performance on six public datasets. [ABSTRACT FROM AUTHOR]

Neural network ensembling is a common and robust way to increase model efficiency. In this paper, we propose a new neural network ensemble algorithm based on Audibert's empirical star algorithm. We provide optimal theoretical minimax bound on the excess squared risk. Additionally, we empirically study this algorithm on regression and classification tasks and compare it to most popular ensembling methods. [ABSTRACT FROM AUTHOR]

Parkinson's disease (PD) is a neurological disease that has been reported to have affected most people worldwide. Recent research pointed out that about 90% of PD patients possess voice disorders. Motivated by this fact, many researchers proposed methods based on multiple types of speech data for PD prediction. However, these methods either face the problem of low rate of accuracy or lack generalization. To develop an approach that will be free of these issues, in this paper we propose a novel ensemble approach. These paper contributions are two folds. First, investigating feature selection integration with deep neural network (DNN) and validating its effectiveness by comparing its performance with conventional DNN and other similar integrated systems. Second, development of a novel ensemble model namely EOFSC (Ensemble model with Optimal Features and Sample Dependant Base Classifiers) that exploits the findings of recently published studies. Recent research pointed out that for different types of voice data, different optimal models are obtained which are sensitive to different types of samples and subsets of features. In this paper, we further consolidate the findings by utilizing the proposed integrated system and propose the development of EOFSC. For multiple types of vowel phonations, multiple base classifiers are obtained which are sensitive to different subsets of features. These features and sample-dependent base classifiers are integrated, and the proposed EOFSC model is constructed. To evaluate the final prediction of the EOFSC model, the majority voting methodology is adopted. Experimental results point out that feature selection integration with neural networks improves the performance of conventional neural networks. Additionally, feature selection integration with DNN outperforms feature selection integration with conventional machine learning models. Finally, the newly developed ensemble model is observed to improve PD detection accuracy by 6.5%. [ABSTRACT FROM AUTHOR]

In this paper, a new method that computes the aesthetics of a melody fragment is proposed, starting from dissonances. While music generated with artificial intelligence applications may be produced considerably more quickly than human-composed music, it has the drawback of not being appreciated like a human composition, being many times perceived by humans as artificial. For achieving supervised machine learning objectives of improving the quality of the great number of generated melodies, it is a challenge to ask humans to grade them. Therefore, it would be preferable if the aesthetics of artificial-intelligence-generated music is calculated by an algorithm. The proposed method in this paper is based on a neural network and a mathematical formula, which has been developed with the help of a study in which 108 students evaluated the aesthetics of several melodies. For evaluation, numerical values generated by this method were compared with ratings provided by human listeners from a second study in which 30 students participated and scores were generated by an existing different method developed by psychologists and three other methods developed by musicians. Our method achieved a Pearson correlation of 0.49 with human aesthetic scores, which is a much better result than other methods obtained. Additionally, our method made a distinction between human-composed melodies and artificial-intelligence-generated scores in the same way that human listeners did. [ABSTRACT FROM AUTHOR]

Deep neural networks are enjoying unprecedented attention and success in recent years. However, catastrophic forgetting undermines the performance of deep models when the training data are arrived sequentially in an online multi-task learning fashion. To address this issue, we propose a novel method named continual learning with declarative memory (CLDM) in this paper. Specifically, our idea is inspired by the structure of human memory. Declarative memory is a major component of long-term memory which helps human beings memorize past experiences and facts. In this paper, we propose to formulate declarative memory as task memory and instance memory in neural networks to overcome catastrophic forgetting. Intuitively, the instance memory recalls the input–output relations (fact) in previous tasks, which is implemented by jointly rehearsing previous samples and learning current tasks as replaying-based methods act. In addition, the task memory aims to capture long-term task correlation information across task sequences to regularize the learning of the current task, thus preserving task-specific weight realizations (experience) in high task-specific layers. In this work, we implement a concrete instantiation of the proposed task memory by leveraging a recurrent unit. Extensive experiments on seven continual learning benchmarks verify that our proposed method is able to outperform previous approaches with tremendous improvements by retaining the information of both samples and tasks. [ABSTRACT FROM AUTHOR]

• A linearly combined form of the objective is reformulated to control the trade-off between the return and the divergence more effectively. • An improved proximal policy optimization method (i.e., alphaPPO) is proposed, with a more elaborative alpha divergence for two adjacent policies. • The effectiveness of our alphaPPO is validated using detailed experimental comparison and analysis for six benchmark environments. Proximal policy optimization (PPO) is a recent advancement in reinforcement learning, which is formulated as an unconstrained optimization problem including two terms: accumulative discount return and Kullback–Leibler (KL) divergence. Currently, there are three PPO versions: primary, adaptive, and clipping. The most widely used PPO algorithm is the clipping version, in which the KL divergence is replaced by a clipping function to measure the difference between two policies indirectly. In this paper, we revisit this primary PPO and improve it in two aspects. One is to reformulate it as a linearly combined form to control the trade-off between two terms. The other is to substitute a parametric alpha divergence for KL divergence to measure the difference of two policies more effectively. This novel PPO variant is referred to as alphaPPO in this paper. Experiments on six benchmark environments verify the effectiveness of our alphaPPO, compared with clipping and combined PPOs. [ABSTRACT FROM AUTHOR]

Since the discovery of the quasicrystal, approximately 100 stable quasicrystals are identified. To date, the existence of quasicrystals is verified using transmission electron microscopy; however, this technique requires significantly more elaboration than rapid and automatic powder X‐ray diffraction. Therefore, to facilitate the search for novel quasicrystals, developing a rapid technique for phase‐identification from powder diffraction patterns is desirable. This paper reports the identification of a new Al–Si–Ru quasicrystal using deep learning technologies from multiphase powder patterns, from which it is difficult to discriminate the presence of quasicrystalline phases even for well‐trained human experts. Deep neural networks trained with artificially generated multiphase powder patterns determine the presence of quasicrystals with an accuracy >92% from actual powder patterns. Specifically, 440 powder patterns are screened using the trained classifier, from which the Al–Si–Ru quasicrystal is identified. This study demonstrates an excellent potential of deep learning to identify an unknown phase of a targeted structure from powder patterns even when existing in a multiphase sample. [ABSTRACT FROM AUTHOR]

The optimal experimental design (OED) provides informative experimental resources and reduces the inherent uncertainty of the experiment. The OED is a framework to maximize performed experiments by controlling design variables. This paper proposes the OED framework combined with deep neural network (DNN) surrogate model to find the optimal design over large and complex design space. The OED has the optimization process to maximize the conditional mutual information (CMI) between hidden properties and composite structural performances. Approximate coordinate exchange (ACE) algorithm was applied to find the optimal design over a high and complicated design space. For fast optimization, surrogate DNN model was used instead of finite element analysis. Two examples were executed for the demonstration of the OED frameworks. From the results of each demonstration, OED provided the results that are consistent with heuristic knowledge. [ABSTRACT FROM AUTHOR]

Soybean (Glycine max (L.) Merr.) is a major source of oil and protein for human food and animal feed; however, soybean crops face diverse factors causing damage, including pathogen infections, environmental shifts, poor fertilization, and incorrect pesticide use, leading to reduced yields. Identifying the level of leaf damage aids yield projections, pesticide, and fertilizer decisions. Deep learning models (DLMs) and neural networks mastering tasks from abundant data have been used for binary healthy/unhealthy leaf classification. However, no DLM predicts and categorizes soybean leaf damage severity (five levels) for tailored pesticide use and yield forecasts. This paper introduces a novel DLM for accurate damage prediction and classification, trained on 2930 near-field soybean leaf images. The model quantifies damage severity, distinguishing healthy/unhealthy leaves and offering a comprehensive solution. Performance metrics include accuracy, precision, recall, and F1-score. This research presents a robust DLM for soybean damage assessment, supporting informed agricultural decisions based on specific damage levels and enhancing crop management and productivity. [ABSTRACT FROM AUTHOR]

In modern wireless telecommunication systems, antenna arrays are widely used as elements of multiple – input multiple – output technology. In the fifth-generation systems, arrays are utilized to realize beamforming that forms the radiation pattern of the base station in the direction of the mobile user. This requires the utilization of many-element antenna arrays that are precisely controlled to achieve the required radiation properties. In this paper we apply the concept of deep neural network to model antenna array radiation properties. In this proof-of-concept research we aim at investigating to what extent it is possible to use deep neural networks for modeling antenna arrays. We consider a full-wave model of linear array with a reflector, which was controlled by the phase and amplitude of the signals feeding the elementary radiators. The applied method made it possible to solve the direct and inverse problems. The results that we obtained show that deep neural networks are able to model antenna array properties. [ABSTRACT FROM AUTHOR]

Vision-based perception modules are increasingly deployed in many applications, especially autonomous vehicles and intelligent robots. These modules are being used to acquire information about the surroundings and identify obstacles. Hence, accurate detection and classification are essential to reach appropriate decisions and take appropriate and safe actions at all times. Current studies have demonstrated that "printed adversarial attacks", known as physical adversarial attacks, can successfully mislead perception models such as object detectors and image classifiers. However, most of these physical attacks are based on noticeable and eye-catching patterns for generated perturbations making them identifiable/detectable by the human eye, in-field tests, or in test drives. In this paper, we propose a camera-based inconspicuous adversarial attack (AdvRain) capable of fooling camera-based perception systems over all objects of the same class. Unlike mask-based FakeWeather attacks that require access to the underlying computing hardware or image memory, our attack is based on emulating the effects of a natural weather condition (i.e., Raindrops) that can be printed on a translucent sticker, which is externally placed over the lens of a camera whenever an adversary plans to trigger an attack. Note, such perturbations are still inconspicuous in real-world deployments and their presence goes unnoticed due to their association with a natural phenomenon. To accomplish this, we develop an iterative process based on performing a random search aiming to identify critical positions to make sure that the performed transformation is adversarial for a target classifier. Our transformation is based on blurring predefined parts of the captured image corresponding to the areas covered by the raindrop. We achieve a drop in average model accuracy of more than 45 % and 40 % on VGG19 for ImageNet dataset and Resnet34 for Caltech-101 dataset, respectively, using only 20 raindrops. [ABSTRACT FROM AUTHOR]

The direct current (DC) microgrid is one of the key research areas for our advancement toward carbon-free energy production. In this paper, a two-step controller is designed for the DC microgrid using a combination of the deep neural network (DNN) and exponential reaching law-based global terminal sliding mode control (ERL-GTSMC). The DC microgrid under consideration involves multiple renewable sources (wind, PV) and an energy storage unit (ESU) connected to a 700 V DC bus and a 4–12 kW residential load. The proposed control method eliminates the chattering phenomenon and offers quick reaching time by utilizing the exponential reaching law (ERL). In the two-step control configuration, first, DNNs are used to find maximum power point tracking (MPPT) reference values, and then ERL-based GTSMC is utilized to track the reference values. The real dynamics of energy sources and the DC bus are mathematically modeled, which increases the system's complexity. Through the use of Lyapunov stability criteria, the stability of the control system is examined. The effectiveness of the suggested hybrid control algorithm has been examined using MATLAB simulations. The proposed framework has been compared to traditional sliding mode control and terminal sliding mode control to showcase its superiority and robustness. Experimental tests based on the hardware-in-the-loop (HIL) setup are then conducted using 32-bit TMS320F28379D microcontrollers. Both MATLAB and HIL results show strong performance under a range of environmental circumstances and system uncertainties. [ABSTRACT FROM AUTHOR]

This paper presents neuro-augmented vision for evolutionary robotics (NAVER), which aims to address the two biggest challenges in camera-equipped robot evolutionary controllers. The first challenge is that camera images typically require many inputs from the controller, which greatly increases the complexity of optimising the search space. The second challenge is that evolutionary controllers often cannot bridge the reality gap between simulation and the real world. This method utilises a variational autoencoder to compress the camera image into smaller input vectors that are easier to manage, while still retaining the relevant information of the original image. Automatic encoders are also used to remove unnecessary details from real-world images, in order to better align with images generated by simple visual simulators. NAVER is used to evolve the controller of a robot, which only uses camera inputs to navigate the maze based on visual cues and avoid collisions. The experimental results indicate that the controller evolved in simulation and transferred to the physical robot, where it successfully performed the same navigation task. The controller can navigate the maze using only visual information. The controller responds to visual cues and changes its behaviour accordingly. NAVER has shown great potential as it has successfully completed (so far) the most complex vision-based task controller in evolutionary robotics literature. [ABSTRACT FROM AUTHOR]

The feasibility of utilizing Deep Neural Networks for generating guidance commands in spacecraft landing scenarios has garnered recent research interest. A primary challenge lies in the inefficiency of training sample generation. This paper proposes a new approach to address it by utilizing nonlinear homotopy methods, which involve embedding the homotopic parameter either into the performance index or the right-hand side of the differential equations. Unlike the existing approaches that overlooked the potential value of the intermediate solutions during homotopy process, the proposed method incorporates these discarded solutions as supplementary training data, which treats the homotopic parameter and state as inputs, and the control as the output. Two examples, namely a minimum-fuel lunar landing and a minimum-time asteroid landing, are provided to illustrate the effectiveness and advantages of the proposed method. • New sample generation method for Deep Neural Networks in spacecraft landing guidance. • Fully utilization of intermediate solutions within the nonlinear homotopy process. • Drastic reduction in the number of optimization problems requiring solving. • Validating the proposed method in lunar and asteroid landing scenarios. [ABSTRACT FROM AUTHOR]

Deficits in social verbal communication in individuals with Social Communication Disorder (SCD) is of concern and SCD in the human community is prevalent in large population throughout the globe. Deficits in verbal social communication are prevalent in a large population. This paper aimed to propose internet connected multi-system architecture which is capable to support verbal communication in a social environment for individuals with social communication deficits. Implementation methodology was included with corpus collection for specific communication, deep learning based machine training for intelligent communication, and implementation of the trained algorithm on internet connected electronic multiple social communication devices. The implemented system is smart enough to initiate and maintain two types of communication; the first type includes communication between multiple individuals on the remote location and the second type includes communication with the individual present in the physical listening range. The system was investigated in terms of its algorithmic parameters and found 97% to 100% in terms of training and testing accuracy with negligible mean squared error. Vocal-Friend analysed results based on audio-bot simulative conditions provide more than 91% accuracy, interaction rate and fallback rate. On the basis of the satisfaction analysis, above average results were noticed. In terms of technical implementations and satisfaction analysis, results found acceptable with above average score. Proposed framework is easy to use by caregivers with even having little knowledge. Support individual with deficit to learn social verbal communication skill to survive in society. Aiding parents, caregivers and professionals to understand the communication needs of individuals with communication deficits. Since technology is also grooming in the domain of rehabilitation, so this system could be used in various future applications such as social robots, social virtual assistants etc. [ABSTRACT FROM AUTHOR]

Intracranial hemorrhage (ICH) is a serious medical condition that necessitates a prompt and exhaustive medical diagnosis. This paper presents a multi-label ICH classification issue with six different types of hemorrhages, namely epidural (EPD), intraparenchymal (ITP), intraventricular (ITV), subarachnoid (SBC), subdural (SBD), and Some. A patient may experience numerous hemorrhages at the same time in some situations. A CT scan of a patient's skull is used to detect and classify the type of ICH hemorrhage(s) present. First, our model determines whether there is a hemorrhage or not; if there is a hemorrhage, the model attempts to identify the type of hemorrhage(s). In this paper, we present a hybrid deep learning approach that combines convolutional neural network (CNN) and Long-Short Term Memory (LSTM) approaches (Conv-LSTM). In addition, to propose viable solutions for the problem, we used a Systematic Windowing technique with a Conv-LSTM. To ensure the efficacy of the proposed model, experiments are conducted on the RSNA dataset. The suggested model provides higher sensitivity (93.87%), specificity (96.45%), precision (95.21%), and accuracy (95.14%). In addition, the obtained F1 score results outperform existing deep neural network-based algorithms. [ABSTRACT FROM AUTHOR]

This paper explores the application of a deep neural network (DNN) framework to human gait analysis for injury classification. The paper aims to identify whether a subject is healthy or has an injury of the ankle, knee, hip, or heel solely based on ground reaction force plate measurements. We consider how three DNNs--the multi-layer perceptron (MLP), fully convolutional network (FCN), and residual network (ResNet)--can be applied to gait analysis when the number of trainable network parameters far exceeds the number of training samples, and benchmark their performance in this context against that of shallow neural networks. The DNN architectures outperformed unsupervised clustering models--self-organizing map and k-means clustering--by a large margin. When tested against support vector machines, which is considered the state-of-the-art approach for supervised gait classification, DNNs performed equal or better despite the propensity for overfitting. We did not find evidence that applying data augmentation to the overfitting problem via timeGAN, a generative adversarial network for time-series data generation, leads to improved classification accuracy. While DNNs are hypothesized to have intrinsic feature extraction capability, the results suggest an advantage to implementing an explicit feature extraction process as a frontend for future applications of deep neural networks: specifically, principal component analysis (PCA) preprocessing improved classification accuracy robustness in all models. In the absence of an explicit feature extraction layer, the use of dropouts in convolutional neural networks caused significant performance degradation; by contrast, the use of a PCA frontend and dropouts synergistically achieved the highest test accuracy among the DNNs studied. [ABSTRACT FROM AUTHOR]

In the fourth industrial revolution, the scale of execution for interactive applications increased substantially. These interactive and animated applications are human-centric, and the representation of human motion is unavoidable, making the representation of human motions ubiquitous. Animators strive to computationally process human motion in a way that the motions appear realistic in animated applications. Motion style transfer is an attractive technique that is widely used to create realistic motions in near real-time. motion style transfer approach employs existing captured motion data to generate realistic samples automatically and updates the motion data accordingly. This approach eliminates the need for handcrafted motions from scratch for every frame. The popularity of deep learning (DL) algorithms reshapes motion style transfer approaches, as such algorithms can predict subsequent motion styles. The majority of motion style transfer approaches use different variants of deep neural networks (DNNs) to accomplish motion style transfer approaches. This paper provides a comprehensive comparative analysis of existing state-of-the-art DL-based motion style transfer approaches. The enabling technologies that facilitate motion style transfer approaches are briefly presented in this paper. When employing DL-based methods for motion style transfer, the selection of the training dataset plays a key role in the performance. By anticipating this vital aspect, this paper provides a detailed summary of existing well-known motion datasets. As an outcome of the extensive overview of the domain, this paper highlights the contemporary challenges faced by motion style transfer approaches. [ABSTRACT FROM AUTHOR]

The exponential growth of social networks has given rise to a wide variety of content. Some social content violates the integrity and dignity of users, therefore, this task has become challenging. The need to deal with short texts, poorly written language, unbalanced classes, and non-thematic aspects. These can lead to overfitting in deep neural network (DNN) models used for classification tasks. Empirical evidence in previous studies indicates that some of these problems can be overcome by improving the optimization process of the DNN weights to avoid overfitting. Moreover, a well-defined learning process in the input examples could improve the order of the patterns learned throughout the optimization process. In this paper, we propose four Curriculum Learning strategies and a new Hybrid Genetic–Gradient Algorithm that proved to improve the performance of DNN models detecting the class of interest even in highly imbalanced datasets. [ABSTRACT FROM AUTHOR]

The lungs are critical components of the respiratory system because they allow for the exchange of oxygen and carbon dioxide within our bodies. However, a variety of conditions can affect the lungs, resulting in serious health consequences. Lung disease treatment aims to control its severity, which is usually irrevocable. The fundamental objective of this endeavor is to build a consistent and automated approach for establishing the intensity of lung illness. This paper describes MixNet-LD, a unique automated approach aimed at identifying and categorizing the severity of lung illnesses using an upgraded pre-trained MixNet model. One of the first steps in developing the MixNet-LD system was to build a pre-processing strategy that uses Grad-Cam to decrease noise, highlight irregularities, and eventually improve the classification performance of lung illnesses. Data augmentation strategies were used to rectify the dataset's unbalanced distribution of classes and prevent overfitting. Furthermore, dense blocks were used to improve classification outcomes across the four severity categories of lung disorders. In practice, the MixNet-LD model achieves cutting-edge performance while maintaining model size and manageable complexity. The proposed approach was tested using a variety of datasets gathered from credible internet sources as well as a novel private dataset known as Pak-Lungs. A pre-trained model was used on the dataset to obtain important characteristics from lung disease images. The pictures were then categorized into categories such as normal, COVID-19, pneumonia, tuberculosis, and lung cancer using a linear layer of the SVM classifier with a linear activation function. The MixNet-LD system underwent testing in four distinct tests and achieved a remarkable accuracy of 98.5% on the difficult lung disease dataset. The acquired findings and comparisons demonstrate the MixNet-LD system's improved performance and learning capabilities. These findings show that the proposed approach may effectively increase the accuracy of classification models in medicinal image investigations. This research helps to develop new strategies for effective medical image processing in clinical settings. [ABSTRACT FROM AUTHOR]

In this paper, we present a web-based optical character recognition (OCR) system that converts images of Ottoman documents printed with naskh font into text using CNN+RNN-based deep neural network models. For training, three datasets - original, synthetic, and hybrid - were prepared and three different OCR models were created. The original data set consists of 1,000 pages and the synthetic data set consists of 23,000 pages. Hybrid data set contains both. The trained models were compared with Tesseract's Arabic and Persian, Google Docs' Arabic, Abby FineReader's Arabic, and Miletos OCR model/tools with a 21-page test set. The comparison was made with 3 different texts (raw, normalized, and joined) and using 3 different criteria (character, ligature, and word recognition). The Osmanlica.com Hybrid model produced significantly better results than the others with 88.86% raw, 96.12% normalized, and 97.37% joined accuracy in character recognition; 80.48% raw, 91.60% normalized, and 97.37% joined accuracy in ligature recognition; and 44.08% raw and 66.45% normalized accuracy in word recognition. To investigate the effects of the characteristics of the alphabet on OCR, character, ligature, and word frequency analyses of Ottoman was performed. In this analysis, the characters in the alphabet were grouped according to distinctive features such as connectedness, letter body, position and number of dots, type of character, and source language; and frequencies and recognition accuracies were examined for each group. OCR results are also reported for each character. [ABSTRACT FROM AUTHOR]

In deep neural network (DNN) accelerators, it is expensive to transfer model parameters from the main memory to the processing elements. Data movement accounts for a large number of the inference latency and energy consumption. In this paper, we present three position-based techniques to compress the DNN model parameters. The techniques could lead to significant energy and performance improvement. The three presented compression techniques are lossless. The first technique takes into consideration the regularly repeat property of the DNN weights to compress them. The second technique saves the relative distance between weights instead of the weights to compress the model. The third technique applies Huffman coding on the relative distance based on the second technique. The proposed techniques are assessed on several DNNs. The results show that, the first technique could decrease 38% of latency and 36% energy, respectively. The second technique could decrease 41% of latency and 39% energy, respectively. The third technique could decrease 45% of latency and 43% energy, respectively. Applying Huffman code could achieve additional 7% reduction in both latency and energy based on the second technique. [ABSTRACT FROM AUTHOR]

Physics-Informed Neural Networks (PINNs) improve the efficiency of data utilization by combining physical principles with neural network algorithms and thus ensure that their predictions are consistent and stable with the physical laws. PINNs open up a new approach to address inverse problems in fluid mechanics. Based on the single-relaxation-time lattice Boltzmann method (SRT-LBM) with the Bhatnagar–Gross–Krook (BGK) collision operator, the PINN-SRT-LBM model is proposed in this paper for solving the inverse problem in fluid mechanics. The PINN-SRT-LBM model consists of three components. The first component involves a deep neural network that predicts equilibrium control equations in different discrete velocity directions within the SRT-LBM. The second component employs another deep neural network to predict non-equilibrium control equations, enabling the inference of the fluid's non-equilibrium characteristics. The third component, a physics-informed function, translates the outputs of the first two networks into physical information. By minimizing the residuals of the physical partial differential equations (PDEs), the physics-informed function infers relevant macroscopic quantities of the flow. The model evolves two sub-models that are applicable to different dimensions, named the PINN-SRT-LBM-I and PINN-SRT-LBM-II models according to the construction of the physics-informed function. The innovation of this work is the introduction of SRT-LBM and discrete velocity models as physical drivers into a neural network through the interpretation function. Therefore, the PINN-SRT-LBM allows a given neural network to handle inverse problems of various dimensions and focus on problem-specific solving. Our experimental results confirm the accurate prediction by this model of flow information at different Reynolds numbers within the computational domain. Relying on the PINN-SRT-LBM models, inverse problems in fluid mechanics can be solved efficiently. [ABSTRACT FROM AUTHOR]

Desynchronization attacks proved to be the greatest challenge to audio watermarking systems as they introduce misalignment between the signal carrier and the watermark. This paper proposes a DNN-based speech watermarking system with two adversarial networks jointly trained on a set of desynchronization attacks to embed a randomly generated watermark. The detector neural network is expanded with spatial pyramid pooling layers to be able to handle signals affected by these attacks. A detailed training procedure of the aforementioned DNN system with gradual attack introduction is proposed in order to achieve robustness. Experiments performed on a speech dataset show that the system achieves satisfactory results according to all the benchmarks it was tested against. The system preserves signal quality after watermark embedding. Most importantly, the system achieved resistance to all considered desynchronization attacks. The majority of the attacks cause less than 1. 7 0 % of incorrectly detected watermarked bits on average, which outperforms comparative techniques in this regard. [ABSTRACT FROM AUTHOR]

This paper proposes a two-step detector called segmented object detection, whose performance is improved by masking the background region. Previous single-stage object detection methods suffer from the problem of imbalance between foreground and background classes, where the background occupies more regions in the image than the foreground. Thus, the loss from the background is firmly incorporated into the training. RetinaNet addresses this problem with Focal Loss, which focuses on foreground loss. Therefore, we propose a method that generates probability maps using instance segmentation in the first step and feeds back the generated maps as background masks in the second step as prior knowledge to reduce the influence of the background and enhance foreground training. We confirm that the detector can improve the accuracy by adding instance segmentation information to both the input and output rather than only to the output results. On the Cityscapes dataset, our method outperforms the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Deep Neural Networks (DNNs) are widely regarded as the most effective learning tool for dealing with large datasets, and they have been successfully used in thousands of applications in a variety of fields. Based on these large datasets, they are trained to learn the relationships between various variables. The adaptive moment estimation (Adam) algorithm, a highly efficient adaptive optimization algorithm, is widely used as a learning algorithm in various fields for training DNN models. However, it needs to improve its generalization performance, especially when training with large-scale datasets. Therefore, in this paper, we propose HN Adam, a modified version of the Adam Algorithm, to improve its accuracy and convergence speed. The HN_Adam algorithm is modified by automatically adjusting the step size of the parameter updates over the training epochs. This automatic adjustment is based on the norm value of the parameter update formula according to the gradient values obtained during the training epochs. Furthermore, a hybrid mechanism was created by combining the standard Adam algorithm and the AMSGrad algorithm. As a result of these changes, the HN_Adam algorithm, like the stochastic gradient descent (SGD) algorithm, has good generalization performance and achieves fast convergence like other adaptive algorithms. To test the proposed HN_Adam algorithm performance, it is evaluated to train a deep convolutional neural network (CNN) model that classifies images using two different standard datasets: MNIST and CIFAR-10. The algorithm results are compared to the basic Adam algorithm and the SGD algorithm, in addition to other five recent SGD adaptive algorithms. In most comparisons, the HN Adam algorithm outperforms the compared algorithms in terms of accuracy and convergence speed. AdaBelief is the most competitive of the compared algorithms. In terms of testing accuracy and convergence speed (represented by the consumed training time), the HN-Adam algorithm outperforms the AdaBelief algorithm by an improvement of 1.0% and 0.29% for the MNIST dataset, and 0.93% and 1.68% for the CIFAR-10 dataset, respectively. [ABSTRACT FROM AUTHOR]

The use of higher frequency bands compared to other wireless communication protocols enhances the capability of accurately determining locations from ultra-wideband (UWB) signals. It can also be used to estimate the number of people in a room based on the waveform of the channel impulse response (CIR) from UWB transceivers. In this paper, we apply deep neural networks to UWB CIR signals for the purpose of estimating the number of people in a room. We especially focus on empirically investigating the various network architectures for classification from single UWB CIR data, as well as from various ensemble configurations. We present our processes for acquiring and preprocessing CIR data, our designs of the different network architectures and ensembles that were applied, and the comparative experimental evaluations. We demonstrate that deep neural networks can accurately classify the number of people within a Line of Sight (LoS), thereby achieving an 99% performance and efficiency with respect to both memory size and FLOPs (Floating Point Operations Per Second). [ABSTRACT FROM AUTHOR]

Differential equations (DEs) are widely employed in the mathematical modelling of a wide range of scientific and engineering problems. The analytical solution of these DEs is typically unknown for a variety of practical problems of relevance. Several numerical methods have been developed over time to find the solution to such DEs and numerous new approaches are still being proposed daily. In recent years, deep learning has emerged as a promising method for solving high-dimensional DEs. Due to the universal approximation capability of neural networks, there is no doubt that studies in this field will continue to grow in the near future. However, there is a need to understand the best-performing neural network architectures and algorithms that demonstrated their effectiveness and ability over traditional algorithms for solving various types of high-dimensional DEs. In this survey, we provide a review of deep learning algorithms classified as artificial neural networks (ANNs) and deep neural networks (DNNs) for solutions of DEs, that have been published in the last decade (between 2011 and 2022). The key purpose of this study is to explore the research papers published in the area of numerical solutions of DEs in order to get a deeper understanding of the current situation. [ABSTRACT FROM AUTHOR]

Over the years, Machine Learning models have been successfully employed on neuroimaging data for accurately predicting brain age. Deviations from the healthy brain aging pattern are associated with the accelerated brain aging and brain abnormalities. Hence, efficient and accurate diagnosis techniques are required to elicit accurate brain age estimations. Several contributions have been reported in the past for this purpose, resorting to different data-driven modeling methods. Recently, deep neural networks (also referred to as deep learning) have become prevalent in manifold neuroimaging studies, including brain age estimation. In this review, we offer a comprehensive analysis of the literature related to the adoption of deep learning for brain age estimation with neuroimaging data. We detail and analyze different deep learning architectures used for this application, pausing at research works published to date quantitatively exploring their application. We also examine different brain age estimation frameworks, comparatively exposing their advantages and weaknesses. Finally, the review concludes with an outlook towards future directions that should be followed by prospective studies. The ultimate goal of this paper is to establish a common and informed reference for newcomers and experienced researchers willing to approach brain age estimation by using deep learning models. • We categorize brain age estimation models based on the deep learning architectures. • We classify deep learning architectures based on the slice/voxel-based input data. • We examine the modalities, size of data, performance and the datasets used. • We enumerate several challenges and trace directions that can be explored further. [ABSTRACT FROM AUTHOR]

The main aim of class discriminative universal adversarial perturbations (CD-UAPs) is that the adversary can flexibly control the targeted class and influence remaining classes limitedly. CD-UAPs generated by the existing attack strategies suffer from a high fooling ratio of non-targeted source classes under non-targeted and targeted attacks, and face the increasing risk of discovery. In this paper, we propose a training framework for generating enhanced covertness CD-UAPs. It trains the targeted source class set and the non-targeted source classes set alternately to update the perturbation and introduces logit pairing to mitigate the influence of perturbation on the non-targeted source classes set. Further, we extend CD-UAPs on the targeted (one-targeted) attack to the multi-targeted attack, which perturbs a targeted source class to multiple targeted sink classes that seriously threaten the current scenario. It can not only provide the adversary with freedom of precise attack but reduce the risk of being detected. This attack poses a strong threat to security-sensitive applications. Extensive experiments on the CIFAR-10, CIFAR-100 and ImageNet datasets show our method can generate more deceptive perturbations and enhance the covertness of CD-UAPs. For example, our method improves the absolute fooling ratio gaps of ResNet-20 and VGG-16 by 9.46% and 6.94% compared with the baseline method, respectively. We achieve the multi-targeted attack with a high fooling ratio on the GTSRB dataset. The average absolute target fooling ratio gaps of ResNet-20 and VGG-16 are 81.89% and 76.33%, respectively. [ABSTRACT FROM AUTHOR]