Search

Modeling and simulating a power distribution network (PDN) for printed circuit boards with irregular board shapes and multi‐layer stackup is computationally inefficient using full‐wave simulations. This paper presents a new concept of using deep learning for PDN impedance prediction. A boundary element method (BEM) is applied to efficiently calculate the impedance for arbitrary board shape and stackup. Then over one million boards with different shapes, stackup, integrated circuits (IC) location, and decap placement are randomly generated to train a deep neural network (DNN). The trained DNN can predict the impedance accurately for new board configurations that have not been used for training. The consumed time using the trained DNN is only 0.1 s, which is over 100 times faster than the BEM method and 10 000 times faster than full‐wave simulations. [ABSTRACT FROM AUTHOR]

Power line inspection is an important means to eliminate hidden dangers of power lines. It is a difficult research problem how to solve the low accuracy of power line inspection based on deep neural network (DNN) due to the problems of multi‐view‐shape, small‐size object. In this paper, an automatic detection model based on Feature visual clustering network (FVCNet) for power line inspection is established. First, an unsupervised clustering method for power line inspection is proposed, and applied to construct a detection model which can recognize multi‐view‐shape objects and enhanced object features. Then, the bilinear interpolation method is used to Feature enhancement method, and the enhanced high‐level semantics and low‐level semantics are fused to solve the problems of small object size and single sample. In this paper, FVCNet is applied to the MS‐COCO 2017 data set and self‐made power line inspection data set, and the test accuracy is increased to 61.2% and 82.0%, respectively. Compared with other models, especially for the categories that are greatly affected by multi‐view‐shape, the test accuracy has been improved significantly. [ABSTRACT FROM AUTHOR]

Summary: One of the most significant factors that influence the globalized economy is agriculture. In order to address the requirement of increasing populations in terms of food necessities, modernizations and technological progressions in agriculture, it is necessary to implement a smart agricultural system. Various traditional techniques are still utilized by the farmers and their intuition in agriculture is not enough to furnish and deliver various issues namely soil management, plant disease identification, weed management, irrigation management, and so forth. Therefore, in this paper, a low‐cost smart farming automation system is evaluated and presented. Since the development of a smart farming automation system minimizes the labor cost and enhances agricultural production level, this paper proposes a deep neural network based Interactive fuzzy Bayesian search (DNN‐IFBS) algorithm for a low‐cost smart farming automation system. In addition to this, the crop water stress index (CWSI) is computed to determine the water status of the plant by employing solar irradiation, canopy temperature, and so forth. The data for analysis is collected simultaneously from three cameras containing image resolutions of about 650 × 460 pixels each. Two different plants namely the melon and sesame are utilized as datasets for experimentation. Finally, the performances of the proposed approach are determined according to the statistical performance measures namely accuracy, specificity as well as F‐measure. The comparative analysis is carried out to evaluate the effectiveness of the proposed system. From the evaluation results, the accuracy rate obtained for the proposed approach is 98.7%. [ABSTRACT FROM AUTHOR]

World's science and technologies have been challenged by the COVID‐19 pandemic. Each and every community across the globe are trying to find a real‐time novel method for accurate treatment and cure of COVID‐19 infected patients. The most important lead to take from this pandemic is to detect the infected patients as soon as possible and provide them an accurate treatment. At present, the worldwide methodology to detect COVID‐19 is reverse transcription‐polymerase chain reaction (RT‐PCR). This technique is costly and time taking. For this reason, the implementation of a novel method is required. This paper includes the use of deep learning analysis to develop a system for identifying COVID‐19 patients. Proposed technique is based on convolution neural network (CNN) and deep neural network (DNN). This paper proposes two models, first is designing DNN on the basis of fractal feature of the images and second is designing CNN using lungs x‐ray images. To find the infected area (tissues) of the lungs image using CNN architecture, segmentation process has been used. Developed CNN architecture gave results of classification with accuracy equal to 94.6% and sensitivity equal to 90.5% which is much better than the proposed DNN method, which gave accuracy 84.11% and sensitivity 84.7%. The outcome of the presented model shows 94.6% accuracy in detecting infected regions. Using this method the growth of the infected regions can be monitored and controlled. The designed model can also be used in post‐COVID‐19 analysis. [ABSTRACT FROM AUTHOR]

This paper trains a multilayered neural network (MLNN) for solving fractional delay differential equations (FDDEs), including nonlinear and singular types. The proposed methodology involves replacing the unknown functions in the equations with a truncated power series expansion. Subsequently, a collection of algebraic equations is solved utilizing an iterative minimization technique that leverages the capabilities of the MLNN architecture. The outcomes demonstrate that the MLNN architecture provides the required accuracy and strong stability compared to several numerical methods. [ABSTRACT FROM AUTHOR]

This paper presents the deep neural network‐based infinitesimal dipole model using either near‐ or far‐field radiation patterns. Based on the radiating characteristic of an infinitesimal dipole, we generated a data set including near and far field radiation patterns corresponding to the phases of infinitesimal dipoles. We used the data set to train and validate the deep neural network (DNN) model. After checking the statistic of the fit function and the average error in expecting the phases of the infinite dipoles, we conclude that the proposed infinitesimal dipole modeling using DNN is sufficient to predict the characteristics of electromagnetic radiation. [ABSTRACT FROM AUTHOR]

One important goal of cognitive science is to understand the mind in terms of its representational and computational capacities, where computational modeling plays an essential role in providing theoretical explanations and predictions of human behavior and mental phenomena. In my research, I have been using computational modeling, together with behavioral experiments and cognitive neuroscience methods, to investigate the information processing mechanisms underlying learning and visual cognition in terms of perceptual representation and attention strategy. In perceptual representation, I have used neural network models to understand how the split architecture in the human visual system influences visual cognition, and to examine perceptual representation development as the results of expertise. In attention strategy, I have developed the Eye Movement analysis with Hidden Markov Models method for quantifying eye movement pattern and consistency using both spatial and temporal information, which has led to novel findings across disciplines not discoverable using traditional methods. By integrating it with deep neural networks (DNN), I have developed DNN+HMM to account for eye movement strategy learning in human visual cognition. The understanding of the human mind through computational modeling also facilitates research on artificial intelligence's (AI) comparability with human cognition, which can in turn help explainable AI systems infer humans' belief on AI's operations and provide human‐centered explanations to enhance human−AI interaction and mutual understanding. Together, these demonstrate the essential role of computational modeling methods in providing theoretical accounts of the human mind as well as its interaction with its environment and AI systems. In this paper, I summarize my research in visual cognition, language processing, and explainable AI to demonstrate the essential role of computational modeling methods in providing theoretical accounts of the human mind as well as its interaction with its environment and AI systems. [ABSTRACT FROM AUTHOR]

Concrete which is the most commercialized construction material and thus it plays a key role in this era of development and hence its evolution is of utmost importance and therefore the evolution of the quality of concrete to that of its highly evolved type namely, ultra‐high performance concrete (UHPC) is undeniably the boon to this sector. Though, the correlations between the technical characteristics of UHPC and the composition of its mixture are complicated, nonlinear, and complex to characterize using standard statistical techniques. This paper is intended to use both deep neural network and ensemble machine learning algorithms namely gradient boosting, extreme gradient boosting, random forest regressor, extra tree regressor, and voting regressor trained with an 810 UHPC mixture collections with 15 input variables to predict its compressive strength. After adjusting a regression model, the prediction efficiency and generalization ability of the developed models are validated using a number of performance parameters. It was established that all employed models performed better at forecasting result, the extra tree regressor was the most accurate followed by extreme gradient boosting. [ABSTRACT FROM AUTHOR]

This paper proposes an efficient method for photovoltaic (PV) system output forecasting by deep Boltzmann machines (DBM) with scatter search‐predator‐prey brain storm optimization (SS‐PPBSO). DBM plays a key role to extract features of input variables while SS‐PPBSO is a new evolutionary computation that combines PPBSO with scatter search. In recent years, as renewable energy, PV systems are positively introduced into power network in Japan so that power system operation becomes complicated due to the uncertainty. To overcome this challenge, it is required to forecast PV outputs that are influenced by weather conditions significantly. This paper proposes a new efficient PV output forecasting method with DBM that makes use of SS‐PPBSO in learning. The effectiveness of the proposed method is demonstrated for real data of a PV system. [ABSTRACT FROM AUTHOR]

The healthcare technologies in COVID‐19 pandemic had grown immensely in various domains. Blockchain technology is one such turnkey technology, which is transforming the data securely; to store electronic health records (EHRs), develop deep learning algorithms, access the data, process the data between physicians and patients to access the EHRs in the form of distributed ledgers. Blockchain technology is also made to supply the data in the cloud and contact the huge amount of healthcare data, which is difficult and complex to process. As the complexity in the analysis of data is increasing day by day, it has become essential to minimize the risk of data complexity. This paper supports deep neural network (DNN) analysis in healthcare and COVID‐19 pandemic and gives the smart contract procedure, to identify the feature extracted data (FED) from the existing data. At the same time, the innovation will be useful to analyse future diseases. The proposed method also analyze the existing diseases which had been reported and it is extremely useful to guide physicians in providing appropriate treatment and save lives. To achieve this, the massive data is integrated using Python scripting language under various libraries to perform a wide range of medical and healthcare functions to infer knowledge that assists in the diagnosis of major diseases such as heart disease, blood cancer, gastric and COVID‐19. [ABSTRACT FROM AUTHOR]

The detection of mathematical expressions is a prerequisite step for the digitisation of scientific documents. Many different multistage approaches have been proposed for the detection of expressions in document images, that is, page segmentation and expression detection. However, the detection accuracy of such methods still needs improvement owing to errors in the page segmentation of complex documents. This paper presents an end‐to‐end framework for mathematical expression detection in scientific document images without requiring optical character recognition (OCR) or document analysis techniques applied in conventional methods. The novelty of this paper is twofold. First, because document images are usually in binary form, the direct use of these images, which lack texture information as input for detection networks, may lead to an incorrect detection. Therefore, we propose the application of a distance transform to obtain a discriminating and meaningful representation of mathematical expressions in document images. Second, the transformed images are fed into the faster region with a convolutional neural network (Faster R‐CNN) optimized to improve the accuracy of the detection. The proposed framework was tested on two benchmark data sets (Marmot and GTDB). Compared with the original Faster R‐CNN, the proposed network improves the accuracies of detection of isolated and inline expressions by 5.09% and 3.40%, respectfully, on the Marmot data set, whereas those on the GTDB data set are improved by 4.04% and 4.55%. A performance comparison with conventional methods shows the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

The driver authority decision for driver‐initiated takeover is closely related to vehicle driving safety. In this paper, a lateral driver‐automation driver authority decision method considering the safety of the intended functionality is proposed to improve vehicle driving safety. Based on systems‐theoretic processes analysis, functional safety analysis of driver initiative takeover is carried out to clarify the safety of the intended functionality issues caused by the unreasonable decision. Then, a method for defining the safe driving area is developed to assess the safety level of the driving area around the ego vehicle. Taking the driver‐vehicle state parameters and vehicle–vehicle state parameters as input, a deep neural network model is constructed to determine the intention of the driver to take over the vehicle. The lateral driver‐automation driver authority decision method is designed to make the driver‐machine take‐over decisions, which considers the safety of the intended functionality. The effectiveness of the proposed method is evaluated via numerical simulation and hardware‐in‐the‐loop experiments. The results show that the designed method not only improves the control ability of the driver to the maximum extent but also integrates the perception ability of the driver into the vehicle control system to improve further vehicle driving safety. [ABSTRACT FROM AUTHOR]

In this paper, we introduce deep learning (DL)‐based backlight extraction methods for local dimming display. The main objective is to better handle the trade‐off between the displayed image quality and power consumption. To this end, we propose an enhanced power regularization. Moreover, we design a smaller model with reduced computational complexity and propose an efficient post‐processing for better generalization. Experimental results show that the proposed methods reduce power consumption while better maintaining the image quality than the selected benchmarks. [ABSTRACT FROM AUTHOR]

Many test coverage metrics have been proposed to measure the deep neural network (DNN) testing effectiveness, including structural coverage and nonstructural coverage. These test coverage metrics are proposed based on the fundamental assumption: They are correlated with test effectiveness. However, the fundamental assumption is still not validated sufficiently and reasonably, which brings question on the usefulness of DNN test coverage. This paper conducted a revisiting study on the existing DNN test coverage from the test effectiveness perspective, to effectively validate the fundamental assumption. Here, we carefully considered the diversity of subjects, three test effectiveness criteria, and both typical and state‐of‐the‐art test coverage metrics. Different from all the existing studies that deliver negative conclusions on the usefulness of existing DNN test coverage, we identified some positive conclusions on their usefulness from the test effectiveness perspective. In particular, we found the complementary relationship between structural and nonstructural coverage and identified the practical usage scenarios and promising research directions for these existing test coverage metrics. [ABSTRACT FROM AUTHOR]

Deep neural network (DNN) surrogates of expensive physics simulations are enabling a rapid change in the way that common experimental design and analysis tasks are approached. Surrogate models allow simulations to be performed in parallel and separately from downstream tasks, thereby enabling analyses that would be impossible with the simulation in‐the‐loop; surrogates based on DNNs can effectively emulate diverse non‐scalar data of the types collected in fusion and laboratory‐astrophysics experiments. The challenge is in training the surrogate model, for which large ensembles of physics simulations must be run, preferably without wasting computational effort on uninteresting simulations. In this paper, we present an iterative sampling scheme that can preferentially propose simulations in interesting regions of parameter space without neglecting unexplored regions, allowing high‐quality and wide‐ranging surrogate models to be trained using 2–3 times fewer simulations compare to space‐filling designs. Our approach uses an explicit importance function defined on the simulation output space, balanced against a measure of simulation density which serves as a proxy for surrogate accuracy. It is easy to implement and can be tuned to find interesting simulations early in the study, allowing surrogates to be trained quickly and refined as new simulations become available; this represents an important step towards the routine generation of deep surrogate models quickly enough to be truly relevant to experimental work. [ABSTRACT FROM AUTHOR]

The speckle noise present in synthetic‐aperture radar (SAR) images is responsible for hindering the extraction of the exact information that needs to be utilized for potential remote sensing applications. Thus the quality of SAR images needs to be enhanced by removing speckle noise in an effective manner. In this paper, A Deep Neural Network‐based Speckle Noise Removal Technique (DNN‐SNRT) is proposed that utilizes the benefits of convolution and Long Short Term Memory‐based neural networks to enhance the quality of SAR images. The proposed DNN‐SNRT uses multiple radar intensity images that are archived from the specific area of interest to facilitate the self‐learning of the intensity features derived from the image patches. The proposed DNN‐SNRT incorporates a dual neural network to remove speckle noise and flexibly estimates the thresholds and weights to achieve an effective SAR image quality improvement. The proposed DNN‐SNRT is capable of automatically updating the intensity features of SAR images during the training process. Experimental investigation of the proposed DNN‐SNRT conducted based on TerraSAR‐X images confirmed the superior enhancement of image quality over comparable recent filters. The results of the DNN‐SNRT scheme were also proved that it is able to reduce noise and preserve edges during the image quality enhancement process. [ABSTRACT FROM AUTHOR]

Given the difficult of experimental determination, quantitative structure–property relationship (QSPR) and deep learning (DL) provide an important tool to predict physicochemical property of chemical compounds. In this paper, partial least squares (PLS), genetic function approximation (GFA), and deep neural network (DNN) were used to predict the Lee retention index (Lee‐RI) of PAHs in SE‐52 and DB‐5 stationary phases. Four molecular descriptors, molecular weight (MW), quantitative estimate of drug‐likeness (QED), atomic charge weighted negative surface area (Jurs_PNSA_3), and relative negative charge (Jurs_RNCG) were selected to construct regression models based on genetic algorithm. For SE‐52, PLS model showed best prediction power, followed by DNN and GFA. The relative error (RE), root mean square error (RMSE), and regression coefficient (R2) of best PLS regression model are 1.228%, 5.407, and 0.980. For DB‐5, DNN model showed best prediction power, followed by GFA and PLS. The RE, RMSE and R2 of best DNN regression model for DB‐5‐1 and DB‐5‐2 are 1.058%, 4.325%, 0.976%, 0.821%, 3.795%, and 0.970%, respectively. The three regression models not only show good predictive ability, but also highlight the stability and ductility of the models. [ABSTRACT FROM AUTHOR]

This paper reported a micro flow cytometer capable of high‐throughput characterization of single‐cell electrical and structural features based on constrictional microchannels and deep neural networks. When single cells traveled through microchannels with constricted cross‐sectional areas, they effectively blocked concentrated electric field lines, producing large impedance variations. Meanwhile, the traveling cells were confined within the cross‐sectional areas of the constrictional microchannels, enabling the capture of high‐quality images without losing focuses. Then single‐cell features from impedance profiles and optical images were extracted from customized recurrent and convolution networks (RNN and CNN), which were further fused for cell‐type classification based on support vector machines (SVM). As a demonstration, two leukemia cell lines (e.g., HL60 vs. Jurkat) were analyzed, producing high‐classification accuracies of 99.3% based on electrical features extracted from Long Short‐Term Memory (LSTM) of RNN, 96.7% based on structural features extracted from Resnet18 of CNN and 100.0% based on combined features enabled by SVM. The microfluidic flow cytometry developed in this study may provide a new perspective for the field of single‐cell analysis. [ABSTRACT FROM AUTHOR]

Summary: In today's world, information security plays a key role in data storage and communication owing to the modern evolution of digitized data interchange in electronic mode. Cryptography is a widely preferred technique for securing transmitted information by transforming the original text into cipher text. A few cryptographic techniques have the inability to protect the data which are vulnerable to a distinct class of attacks. Therefore, a reliable cryptographic technique is necessitated for enlarging information security. In this paper, a new hybrid cryptanalysis (HCA) model has been proposed to acquire optimal cryptanalysis of data encryption standard (DES). It combines linear cryptanalysis (LCA) and neural cryptanalysis (NCA) for enhancing the performance of the cryptographic system with minimal time complexity. Primarily, the HCA model employs LCA to break cryptographic codes by analyzing linear approximations of the cryptographic algorithm. NCA is then applied with the aid of a deep neural network for identifying patterns in larger datasets. These patterns can be further utilized to break encryption. The efficacy of the proposed HCA model will be assessed through the assessment of three different datasets. The results manifest that the accuracy of the proposed model is increased up to 97.66% and the time needed to break encryption can be reduced. [ABSTRACT FROM AUTHOR]

High‐dimensional classification is a fundamentally important research problem in high‐dimensional data analysis. In this paper, we derive a nonasymptotic rate for the minimax excess misclassification risk when feature dimension exponentially diverges with the sample size and the Bayes classifier possesses a complicated modular structure. We also show that classifiers based on deep neural networks can attain the above rate, hence, are minimax optimal. [ABSTRACT FROM AUTHOR]

Price impact is the adverse change of the asset price against trader's action. As a crucial part of the indirect trading cost, price impact has attracted increasing attention in both econometric and data science literature. In this paper, we draw upon both strands of the literature and develop a deep neural network enhanced recursive (DeRecv) model to estimate temporary and permanent price impact of an order or trade. The temporary price impact is calculated as the sum of the expected immediate impact at each time point after taking action in an ad hoc market condition. The permanent price impact is defined as a new permanent level at which the information of the incoming order is entirely absorbed by the market. Through the experimental evaluation based on data from 10 stocks at NASDAQ and Shanghai Stock Exchange, we show that the proposed DeRecv model is better than the reinforcement learning model and the traditional vector autoregressive model. [ABSTRACT FROM AUTHOR]

Summary: Deep neural networks have been demonstrated to be useful in varieties of intelligent tasks, and various specialized NN accelerators have been proposed recently to improve the hardware efficiency, which are typically equipped with software‐managed scratchpad memory (SPM) for high performance and energy efficiency. However, traditional SPM management techniques cause memory fragmentation for NN accelerators, and thus lead to low utilization of precious SPM. The main reason is that traditional techniques are originally designed for managing fixed‐length registers rather than variable‐length memory blocks. In this article, we propose a novel SPM management approach for NN accelerators. The basic intuition is that NN computation/memory behaviors are predictable and relatively regular compared with traditional applications, and thus most information can be determined at compile time. In addition, by exploiting the variable‐length feature of SPM, we propose to divide the allocation process into two passes: the space assignment and the address assignment pass, which are simultaneously (and implicitly) performed in traditional one‐pass allocation techniques. Experimental results on the memory requests of a representative NN accelerator demonstrate that the proposed approach can significantly reduce the memory consumption by 30% at most compared with state‐of‐the‐art SPM management techniques, and the memory usage is only 2% larger than that of the theoretical optimal allocation. [ABSTRACT FROM AUTHOR]

Inverse modeling of microwave components plays an important role in microwave design and diagnosis or tuning. Since the analytical function or formula of the inverse input‐output relationship does not exist and is difficult to obtain, artificial neural network (ANN) becomes an efficient tool to develop inverse models for microwave components. This paper provides an overview of recent advances in neural network‐based inverse modeling techniques for microwave applications. We review two different shallow neural network‐based inverse modeling techniques, including the comprehensive neural network inverse modeling methodology and the multivalued neural network inverse modeling technique. Both techniques address the problem of nonuniqueness in inverse modeling. We also provide an overview of recently developed hybrid deep neural network modeling technique and the application to inverse modeling. For the inverse modeling problem with high‐dimensional inputs, the relationship between the inputs and the outputs of the inverse model will become more complicated and the inverse modeling problem will become harder. The deep neural network becomes a practical choice. The hybrid deep neural network structure is presented. The recently proposed activation function, specifically for microwave application, and a three‐stage deep learning algorithm for training the hybrid deep neural network are reviewed. [ABSTRACT FROM AUTHOR]

The syndrome called COVID‐19 which was firstly spread in Wuhan, China has already been declared a globally "Pandemic." To stymie the further spread of the virus at an early stage, detection needs to be done. Artificial Intelligence‐based deep learning models have gained much popularity in the detection of many diseases within the confines of biomedical sciences. In this paper, a deep neural network‐based "LiteCovidNet" model is proposed that detects COVID‐19 cases as the binary class (COVID‐19, Normal) and the multi‐class (COVID‐19, Normal, Pneumonia) bifurcated based on chest X‐ray images of the infected persons. An accuracy of 100% and 98.82% is achieved for binary and multi‐class classification respectively which is competitive performance as compared to the other recent related studies. Hence, our methodology can be used by health professionals to validate the detection of COVID‐19 infected patients at an early stage with convenient cost and better accuracy. [ABSTRACT FROM AUTHOR]

Purpose: Fetal head circumference (HC) is an important biometric parameter that can be used to assess fetal development in obstetric clinical practice. Most of the existing methods use deep neural network to accomplish the task of automatic fetal HC measurement from two‐dimensional ultrasound images, and some of them achieved relatively high prediction accuracy. However, few of these methods focused on optimizing model efficiency performance. Our purpose is to develop a more efficient approach for this task, which could help doctors measure HC faster and would be more suitable for deployment on devices with scarce computing resources. Methods: In this paper, we present a very lightweight deep convolutional neural network to achieve automatic fetal head segmentation from ultrasound images. By using sequential prediction network architecture, the proposed model could perform much faster inference while maintaining a high prediction accuracy. In addition, we used depthwise separable convolution to replace part of the standard convolution in the network and shrunk the input image to further improve model efficiency. After getting fetal head segmentation results, post‐processing, including morphological processing and least‐squares ellipse fitting, was applied to obtain the fetal HC. All experiments in this work were performed on a public dataset, HC18, with 999 fetal ultrasound images for training and 335 for testing. The dataset is publicly available on https://hc18.grand‐challenge.org/ and the code for our method is also publicly available on https://github.com/ApeMocker/CSM‐for‐fetal‐HC‐measurement. Results: Our model has only 0.13 million [M] parameters and can achieve an inference speed of 28 [ms] per frame on a CPU and 0.194 [ms] per frame on a GPU, which far exceeds all existing deep learning‐based models as far as we know. Experimental results showed that the method achieved a mean absolute difference of 1.97(± 1.89) [mm] and a Dice similarity coefficient of 97.61(± 1.72) [%] on HC18 test set, which were comparable to the state of the art. Conclusion: We presented a very lightweight deep learning‐based model to realize fast and accurate fetal head segmentation from two‐dimensional ultrasound image, which is then used for calculating the fetal HC. The proposed method could help obstetricians measure the fetal HC more efficiently with high accuracy, and has the potential to be applied to the situations where computing resources are relatively scarce. [ABSTRACT FROM AUTHOR]

Summary: Multiple‐input multiple‐output (MIMO) technology is much significant for achieving high data rates while considering the multiuser communication. For this purpose, the estimation of Channel State Information (CSI) is performed by the pilot. But, the estimated CSI cannot be applied for downlinks while processing an extremely high speed mobile data as it gets outdated because of the rapid channel variation. Estimating the channel in the MIMO systems is highly difficult on using non‐stationary channel characteristics under the high mobility environments. Channel estimation with received signal‐to‐noise ratio (SNR) feedback ensures an effective performance in the practical wireless MIMO systems. This paper plans to develop an intelligent deep learning strategy to solve the channel estimation problem in MIMO system. The main goal of the proposed model is to estimate the MIMO channel coefficients at a transmitter based on the received SNR feedback information from a receiver using the swarm intelligence‐based deep ensemble learning machine (SI‐DELM). For both the scenarios, the SI‐DELM is developed, in which the convolution neural network (NN) with three classifiers is used. Here, the fully connected layers of CNN are replaced by the NN, recurrent neural network (RNN), and deep neural network (DNN). In the SI‐DEL‐based channel estimation, the hyper parameter tuning is performed by the adaptive iteration assisted mixture ratio‐based cat swarm optimization (AIMR‐CSO) for minimizing the mean square error (MSE) and bit error rate (BER) of the estimated channel. Finally, by performing numerical simulations, the efficiency and better performance of the suggested model is revealed by conducting the comparison between proposed and conventional models. [ABSTRACT FROM AUTHOR]

Automatic and robust target depth estimation is an important issue of an active detection system working in the reliable acoustic path (RAP) environment. In this paper, the target depth‐sensitive acoustic interference structure is used as input, and a deep neural network (DNN) method is proposed to realize automatic depth estimation. Furthermore, to improve the robustness of the network, both the sound speed profile (SSP) uncertainty and the target trajectory uncertainty are considered in the training dataset. The former is modelled as a random weighted sum of the empirical orthogonal function (EOF), and the latter is modelled by the partial‐data‐lacking of the network input. The method was evaluated through simulated data and compared with the conventional method of matching the measured interference structure with the replicas. The presented method exhibits a higher accuracy of depth estimate in uncertain environments when the signal‐to‐noise ratio (SNR) is higher than −6 dB, but its performance deteriorates rapidly as the SNR decreases further. The method is also verified by semi‐simulation with experimental data in deep water. [ABSTRACT FROM AUTHOR]

This paper describes the development of three‐dimensional (3D) human recognition system of flying drone system for emergency rescue and investigation. In this system, deep neural network and its application for 3D object recognition are key techniques for human detection from a free viewpoint. Some appearance based characteristics are captured as movie frames, and the system uses deep neural networks to automatically classify concerned object. The proposed system performs well that many kinds of views of personnel can be recognized from bird's eye view. Experimental results show that the system can effectively recognize human objects with high accuracy. [ABSTRACT FROM AUTHOR]

Although more layers and more parameters generally improve the accuracy of the models, such big models generally have high computational complexity and require big memory, which exceed the capacity of small devices for inference and incurs long training time. In addition, it is difficult to afford long training time and inference time of big models even in high performance servers, as well. As an efficient approach to compress a large deep model (a teacher model) to a compact model (a student model), knowledge distillation emerges as a promising approach to deal with the big models. Existing knowledge distillation methods cannot exploit the elastic available computing resources and correspond to low efficiency. In this paper, we propose an Elastic Deep Learning framework for knowledge Distillation, that is, EDL‐Dist. The advantages of EDL‐Dist are threefold. First, the inference and the training process is separated. Second, elastic available computing resources can be utilized to improve the efficiency. Third, fault‐tolerance of the training and inference processes is supported. We take extensive experimentation to show that the throughput of EDL‐Dist is up to 3.125 times faster than the baseline method (online knowledge distillation) while the accuracy is similar or higher. [ABSTRACT FROM AUTHOR]

In this paper, the geometry factors of mode I and mode II (YI and YII) and the normalized T‐stress (T*) of the centrally cracked Brazilian disk specimen are predicted using a deep learning approach. Three deep neural networks are developed to model the relationship between the crack angle (α) and the ratio of half the crack length to the radius (a/R) as input variables and each of YI, YII, and T* as output variables. Three independent databases consisting of 174, 174, and 117 data points are prepared for YI, YII, and T*, respectively, from the previous works to train, generalize, and validate the deep neural networks. Finally, sensitivity analysis of YI, YII, and T* to α and a/R is conducted, and mathematical models are obtained using the partial dependence plots, which can be used in the optimization of stress intensity factors and T‐stress of the cracked Brazilian disk specimens with different dimensions. Highlights: Three analytical databases are prepared for YI, YII, and T*.Three independent DNN models are developed for predicting YI, YII, and T* of the BD specimen.The sensitivity of YI, YII, and T* to the α and a/R are evaluated using PDP analysis.The mathematical relationship between α and a/R, and YI, YII, and T* are obtained. [ABSTRACT FROM AUTHOR]

Spintronic devices are considered as one of the most promising technologies for non‐volatile memory and computing. However, two crucial drawbacks, that is, lack of intrinsic multi‐level operation and low on/off ratio, greatly hinder their further application for advanced computing concepts, such as deep neural network (DNN) accelerator. In this paper, a spintronic multi‐level memory unit with high on/off ratio is proposed by integrating several series‐connected magnetic tunnel junctions (MTJs) with perpendicular magnetic anisotropy (PMA) and a Schottky diode in parallel. Due to the rectification effect on the PMA MTJ, an on/off ratio over 100, two orders of magnitude higher than intrinsic values, is obtained under proper proportion of alternating current and direct current. Multiple resistance states are stably achieved and can be reconfigured by spin transfer torque effect. A computing‐in‐memory architecture based DNN accelerator for image classification with the experimental parameters of this proposal to evidence its application potential is also evaluated. This work can satisfy the rigorous requirements of DNN for memory unit and promote the development of high‐accuracy and robust artificial intelligence applications. [ABSTRACT FROM AUTHOR]

Visual interpretation of chest X‐rays (CXRs) is tedious and prone to error. Significant amount of time is spent by the radiologist in differentiating normal from abnormal CXRs and in identifying the location and type of abnormalities. An assistance tool for automatically classifying normal and different types of abnormal CXRs can facilitate the diagnosis and potentially save time costs. In this paper, a novel hybrid model having concatenation of Visual Geometry Group (VGG19) network and Entropy features as a modified deep convolutional neural network (DCNN) architecture, called VEntNet, is proposed for the automated multi‐class categorization of CXR images into normal, coronavirus disease (COVID), tuberculosis (TB), viral pneumonia, and bacterial pneumonia. The VEntNet model implemented consists of deep features extraction from convolutional layers of VGG19 network which are then concatenated with hand‐crafted entropy features extracted from CXRs. The concatenated features are then fed to the fully connected (FC) layers for performing multi‐class categorization using Softmax activation function. The performance of proposed VEntNet model is compared with other DCNNs with and without the hybrid approach for categorization of closely related lung pathologies and normal CXR images. Our proposed VEntNet achieved accuracies of 98.78% and 90.96%, respectively, for four and five‐class classification of CXRs. Thus, it is demonstrated that among the different DCNNs, our VEntNet outperformed in four‐class CXR categorization tasks. The proposed model can potentially save time by facilitating the screening of CXRs to identify those with abnormalities present as well as to categorize the abnormalities. [ABSTRACT FROM AUTHOR]

Compared with AC distribution networks, DC ones have a number of advantages. Intensive connections of distributed renewable energy can lead to large amount of power electronic converters and complex models. Underground cable is widely used in DC distribution networks. Accurate location of faults can help engineers find the fault points and shorten the time of maintenance. In DC distribution networks, where only a few measuring units are equipped and low sampling rates are adopted, there is limited data used for fault location. Also, for monopole grounding fault, the fault features are sometimes unobvious for recognition. Deep learning which provides feature hierarchy can learn experiences automatically and recognise raw data as human brain does. It reveals a high potential to solve location problems in DC distribution systems. This paper proposes a depth learning based fault location for DC distribution networks. First, a DC distribution network with radiant topology is modelled, and faults are added with different parameters to simulate various scenarios in practical projects. Then, a deep neural network is generated and trained with normalised fault currents. The parameters of network are discussed according to particular application. Finally, the location performance of deep neural network is tested. [ABSTRACT FROM AUTHOR]

In this paper, a nonlocal energy‐informed neural network is proposed to characterize the deformation behaviors of elastic solids with cracks subjected to thermomechanical loads in the framework of ordinary state‐based peridynamics. Based on principle of virtual work, a nonlocal energy approach is developed to recast the solution to peridynamic equilibrium equation as a problem of minimizing the potential energy of the system, which automatically satisfies the traction‐free boundary conditions. Meanwhile, the energy representation of physical system can be treated as the loss function for machine learning methods. Therefore, a nonlocal energy‐informed neural network is constructed to approximate the solution of the system. A distinct advantage of the proposed neural network is that the strain energy is expressed in terms of spatial integration instead of spatial derivatives, which avoids the invalidation of automatic differentiation at the crack surfaces in original physics‐informed neural networks. To demonstrate the convergence and accuracy of the proposed neural network, a series of problems in solid and fracture mechanics are conducted, and compared with results from analytical solutions or classical numerical methods. Additionally, for elastic material containing initial cracks, displacement extrapolation method is encoded into the proposed neural network to evaluate the static stress intensity factor. [ABSTRACT FROM AUTHOR]

Summary: Various deep neural network (DNN) model watermarks have been proposed by researchers to verify copyrights for deep neural networks DNN. However, most DNN watermarking methods cannot prevent attackers from stealing and using the model. Unlike many existing approaches, this paper uses a channel pruning algorithm to protect DNN models, which verifies DNN models copyrights but also prevents the illegal use of DNN models. In this work, the pruning threshold or pruning rate is used as the secret key of a DNN model. After the secret key is distributed to multiple users, they prune the DNN model with the secret key, and the pruned and fine‐tuned model is provided to the users. The users can verify ownership of the model according to the pruning accuracy and fine‐tuning accuracy. If the secret key is incorrect, the accuracy of the model after fine‐tuning will be very low, and users will be unable to use the reasoning function of the fine‐tuned model. Based on the CIFAR‐10 and CIFAR‐100 datasets, we conducted experiments on five popular DNN models. The experimental results show that we can authorize multiple users by pruning very few channels in the convolution layers of the DNN model. [ABSTRACT FROM AUTHOR]

With the recent advances achieved by deep neural networks in image processing applications, researchers have begun exploring deep learning in pansharpening and obtained remarkable results. However, the existing methods are generally limited by their weak feature representation ability, often leading to spectral distortion or spatial blur. To generate high‐quality pansharpened images, this article proposes a novel neural network for pansharpening that includes both feature extraction and excitation mechanisms to consider important features. The neural network is modified with domain knowledge in pansharpening to fully extract spectral and spatial structures, and the proposed method outperforms traditional methods. [ABSTRACT FROM AUTHOR]

We contribute in the current paper to extend the universal function approximation property of deep learning hybrid strategy to design both : (a) adaptive deep neural network observer to estimate derivatives of the tracking error dynamics (b) and robust deep neural network (DNN) output feedback control scheme (OFCS) that will overcome successfully effects of both parametric variations and modeling errors. First, the designed controller employs OFCS to linearize the partially known antilock bracking (ABS) dynamics, and then, the dynamic compensator is involved to stabilize the linearized system. However, conventional controllers suffer from limitations due to the presence of these high uncertainties. Therefore, we aim to demonstrate for the first time in the control area the feasibility of applying deep learning algorithm based on sparse auto‐encoder as an approximator for neglected dynamics and uncertain parameters of the ABS. The estimated states are used as inputs to the neural network (NN) and in the adaptation laws as an error signal. Simulations of the proposed control algorithm based adaptive DNN observer are conducted then compared to bang–bang controller, PI controller, and adaptive controller‐based single hidden layer‐neural network with only one‐neuron in hidden layers (SHL(1N)NN) to demonstrate its practical potential. Furthermore, both feasibility and efficiency of involving deep learning algorithm (DLA) in the control area have been successfully confirmed through robustness test. [ABSTRACT FROM AUTHOR]

Fouling degrades the thermal and hydraulic performances of the heat exchanger (HE), leading to failure if undetected. It occurs due to the accumulation of undesired material on the heat transfer surface. Knowledge about the HE fouling dynamics is required to plan mitigation strategies, ensuring a sustainable and safe operation. This paper aims to propose a feature‐based technique to predict the fouling status of the HE based on historical data. Three thermal and two hydraulic features are extracted from the HE. Random forest (RF) is employed to detect the dominant features using the Gini index. These dominant features are used to predict the fouling resistance using a deep neural network based on the Long Short‐Term Memory (LSTM) model. Also, these dominant features provide reliable inferences to reason out the fouling dynamics. A diagnostic flag is derived based on the dominant feature and is used to diagnose the ongoing fouling phenomena, which is vital to formulate mitigation. The proposed technique is investigated on the data acquired from the HEs used in a thermal power plant (Case 1) and petroleum refinery (Case 2). Prediction accuracy of 99% and 97% is observed for Cases 1 and 2, respectively. Experimental results illustrate that RF enables the LSTM to achieve faster training and reliable prediction of fouling. [ABSTRACT FROM AUTHOR]

Predicting the performance of thermoelectric generators (TEGs) is an essential part of designing high‐performance TEGs. However, due to the complexity of the TEG system, the existing methods are either time‐consuming or not precise enough, inconvenient for device optimization. In this paper, the deep learning (DL) method to fast and accurately get the performance of TEG devices is presented. First, the key features of a typical TEG device are captured and the training dataset is prepared based on the extracted features and finite element simulations. Next, a proper deep neural network architecture is acquired and the model is trained to converge at a low loss. Finally, the experimental data is used to validate the generalization ability of the presented model. Besides, the device optimization based on the DL solution is performed and an output power enhancement of up to 182% is achieved for the authors' sample module. The presented DL solution thus can be well applied in designing or optimizing high‐performance TEGs. Furthermore, the established framework also sheds considerable light on applying the DL approach to solve general engineering problems. [ABSTRACT FROM AUTHOR]

Traditional Eulerian-based fluid simulations require much time and computational resources to solve the projection step, especially the large linear system produced by the Poisson equation. In this paper, we propose an adaptive machine-learning-based projection method combining deep neural network and incremental learning technique. We provide two modes: Fast Mode and Normal Mode to solve the most time-consuming projection step and deal with various simulation scenes largely different from the training data set. We introduce patch-based feature vectors to represent the whole velocity field and a loss function to keep the divergence-free constraint. We have demonstrated the acceleration and extrapolation ability of our method by testing various smoke scenes far different from our training data set. [ABSTRACT FROM AUTHOR]

Since the large‐scale protein sequence data is available, applying deep neural networks to mine better features from the sequences becomes possible. Eukaryotic protein subcellular localization prediction which makes a contribution in many biology process, has used protein sequences in many automatic predicting methods. Moreover, gene ontology (GO) annotation has been shown to be helpful in improving the prediction accuracy of subcellular localization. However, experimentally annotated proteins are not always available. On the other hand, experimentally annotated proteins are available for certain species such as human, mouse, Arabidopsis thaliana, etc. It is highly motivated to perform deep learning of GO annotations on the available experimentally annotated proteins and to transfer it to subcellular localization prediction on other species. In this paper, we propose a deep protein subcellular localization predictor, consisting of a linear classifier and a deep feature extractor of convolution neural network (CNN). The deep CNN feature extractor is first shared and pre‐trained in a deep GO annotation predictor, and then is transferred to the subcellular localization predictor with fine‐tuning using protein localization samples. In this way, we have a deep protein subcellular localization predictor enhanced with transfer learning of GO annotation. The proposed method has good performances on the Swiss‐Prot datasets, when transfer learning using the protein samples both within and out species. Moreover, it outperforms the state‐of‐the‐art traditional methods on benchmark datasets. © 2021 Institute of Electrical Engineers of Japan. Published by Wiley Periodicals LLC. [ABSTRACT FROM AUTHOR]

Neural networks have generated valuable Quantitative Structure-Activity/Property Relationships (QSAR/QSPR) models for a wide variety of small molecules and materials properties. They have grown in sophistication and many of their initial problems have been overcome by modern mathematical techniques. QSAR studies have almost always used so-called 'shallow' neural networks in which there is a single hidden layer between the input and output layers. Recently, a new and potentially paradigm-shifting type of neural network based on Deep Learning has appeared. Deep learning methods have generated impressive improvements in image and voice recognition, and are now being applied to QSAR and QSAR modelling. This paper describes the differences in approach between deep and shallow neural networks, compares their abilities to predict the properties of test sets for 15 large drug data sets (the kaggle set), discusses the results in terms of the Universal Approximation theorem for neural networks, and describes how DNN may ameliorate or remove troublesome 'activity cliffs' in QSAR data sets. [ABSTRACT FROM AUTHOR]

In this paper, we propose a rank‐weighted reconstruction feature to improve the robustness of a feed‐forward deep neural network (FFDNN)‐based acoustic model. In the FFDNN‐based acoustic model, an input feature is constructed by vectorizing a submatrix that is created by slicing the feature vectors of frames within a context window. In this type of feature construction, the appropriate context window size is important because it determines the amount of trivial or discriminative information, such as redundancy, or temporal context of the input features. However, we ascertained whether a single parameter is sufficiently able to control the quantity of information. Therefore, we investigated the input feature construction from the perspectives of rank and nullity, and proposed a rank‐weighted reconstruction feature herein, that allows for the retention of speech information components and the reduction in trivial components. The proposed method was evaluated in the TIMIT phone recognition and Wall Street Journal (WSJ) domains. The proposed method reduced the phone error rate of the TIMIT domain from 18.4% to 18.0%, and the word error rate of the WSJ domain from 4.70% to 4.43%. [ABSTRACT FROM AUTHOR]

Dysarthria is a degenerative disorder of the central nervous system that affects the control of articulation and pitch; therefore, it affects the uniqueness of sound produced by the speaker. Hence, dysarthric speaker recognition is a challenging task. In this paper, a feature‐extraction method based on deep belief networks is presented for the task of identifying a speaker suffering from dysarthria. The effectiveness of the proposed method is demonstrated and compared with well‐known Mel‐frequency cepstral coefficient features. For classification purposes, the use of a multi‐layer perceptron neural network is proposed with two structures. Our evaluations using the universal access speech database produced promising results and outperformed other baseline methods. In addition, speaker identification under both text‐dependent and text‐independent conditions are explored. The highest accuracy achieved using the proposed system is 97.3%. [ABSTRACT FROM AUTHOR]

Summary: The objective of this paper is twofold. First, it develops a prediction system to help the credit card issuer model the credit card delinquency risk. Second, it seeks to explore the potential of deep learning (also called a deep neural network), an emerging artificial intelligence technology, in the credit risk domain. With real‐life credit card data linked to 711,397 credit card holders from a large bank in Brazil, this study develops a deep neural network to evaluate the risk of credit card delinquency based on the client's personal characteristics and the spending behaviours. Compared with machine‐learning algorithms of logistic regression, naive Bayes, traditional artificial neural networks, and decision trees, deep neural networks have a better overall predictive performance with the highest F scores and area under the receiver operating characteristic curve. The successful application of deep learning implies that artificial intelligence has great potential to support and automate credit risk assessment for financial institutions and credit bureaus. [ABSTRACT FROM AUTHOR]

It has been reported that training deep neural networks is more difficult than training shallow neural networks. Hinton et al. proposed deep belief networks with a learning algorithm that trains one layer at a time. A much better generalization can be achieved when pre‐training each layer with an unsupervised learning algorithm. Since then, deep neural networks have been extensively studied. On the other hand, it has been revealed that singular points affect the training dynamics of the learning models such as neural networks and cause a standstill of training. Naturally, training deep neural networks suffer singular points. As described in this paper, we present a deep neural network model that has fewer singular points than the usual one. First, we demonstrate that some singular points in the deep real‐valued neural network, which is equivalent to a deep complex‐valued neural network, have been resolved as its inherent property. Such deep neural networks are less likely to become trapped in local minima or plateaus caused by critical points. Results of experiments on the two spirals problem, which has an extreme nonlinearity, support our theory. Copyright © 2017 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

Satellite‐based remote sensing missions have revolutionized our understanding of the Ocean state and dynamics. Among them, space‐borne altimetry provides valuable Sea Surface Height (SSH) measurements, used to estimate surface geostrophic currents. Due to the sensor technology employed, important gaps occur in SSH observations. Complete SSH maps are produced using linear Optimal Interpolations (OI) such as the widely used Data Unification and Altimeter Combination System (duacs). On the other hand, Sea Surface Temperature (SST) products have much higher data coverage and SST is physically linked to geostrophic currents through advection. We propose a new multi‐variate Observing System Simulation Experiment (OSSE) emulating 20 years of SSH and SST satellite observations. We train an Attention‐Based Encoder‐Decoder deep learning network (abed) on this data, comparing two settings: one with access to ground truth during training and one without. On our OSSE, we compare abed reconstructions when trained using either supervised or unsupervised loss functions, with or without SST information. We evaluate the SSH interpolations in terms of eddy detection. We also introduce a new way to transfer the learning from simulation to observations: supervised pre‐training on our OSSE followed by unsupervised fine‐tuning on satellite data. Based on real SSH observations from the Ocean Data Challenge 2021, we find that this learning strategy, combined with the use of SST, decreases the root mean squared error by 24% compared to OI. Plain Language Summary: The surface of the ocean is observed through various sensors embedded in satellites. Specifically, the height of the sea surface is a crucial variable as it can be used to estimate surface currents. It is currently measured through satellite altimeters, but the data acquisition process leaves gaps in their observations. Providing fully gridded maps of the sea surface height is thus an important interpolation problem. The widely used interpolated product has some troubles, especially when dealing with small and rapidly evolving eddies. To enhance its quality, we propose an artificial neural network, a trainable method able to estimate complete sea surface height images. The flexibility of these methods allows us to use different satellite information, such as the sea surface temperature, which has a better resolution. Usually, neural networks are trained on a data set upon which they learn the link between input and output data. However, in a realistic geoscience scenario, the output is never known for sure. We propose a data set that simulates this problem and explores methodologies to train these methods. Finally, we design a new way of learning SSH reconstruction using both simulated data and real satellite observation. Key Points: We developed a realistic simulation of satellite observations of sea surface height and temperatureWe compare deep learning supervised and unsupervised strategies to interpolate the sea surface heightWe present a hybrid learning strategy using supervised training on our simulated data and unsupervised fine‐tuning on real‐world data [ABSTRACT FROM AUTHOR]

In the big data network environment, because traditional user abnormal behavior detection methods cannot meet the needs of massive data detection, it cannot quickly respond to constantly updated abnormal behaviors and malware, and does not consider user behavior management. The accuracy of abnormal detection and stability are insufficient. By combining with network traffic analysis technology, this paper propose a custom user abnormal behavior detection model based on deep neural networks which implements fine‐grained analysis of network traffic and customizes user behavior management settings to enable user abnormal detection to meet the requirements of specific network environments. The custom user abnormal behavior detection model uses the data of network traffic analysis as the input vector of the deep neural network algorithm to detect unknown abnormal behaviors in massive data and implement custom user behavior management. Experimental results show that the proposed method has high accuracy, can effectively implement custom user behavior management, and overcomes the shortcomings of traditional user abnormal behavior detection methods. [ABSTRACT FROM AUTHOR]

This paper proposes a framework for correcting the actions during sport training. We estimate the positions of bone joint points and the postures by using a convolutional neural network. The positions of bone joint points and the postures are the premise to correct the actions during sport training. In convolutional neural network, it first predicts the positions of bone joint points via confidential map of different parts; then utilizes affinity domain, which is a 2‐dimensional vector set, to predict the directions and positions of four limbs; lastly repeats the above steps and analyzes the bone joint points via greedy analytical reasoning algorithm to construct the human skeleton. The personal action is represented as the constructed skeleton which is used to correct the incorrect or non‐standard actions during sports. The experimental results evaluate the effective of the proposed sport training action correction method. [ABSTRACT FROM AUTHOR]

Artificial intelligence (AI) represents a rapidly developing field. Its use can improve diagnosis and therapy in many areas of medicine. Despite this enormous progress, many physicians perceive it as a black box and are skeptical about it. This review will present the basics of machine learning. Different classifications of artificial intelligence, such as supervised versus unsupervised and discriminative versus generative AI, are given. Analogies to human intelligence are discussed as far as algorithms are oriented toward it. In the second step, the most common models like random forest, k‐means clustering, convolutional neural network, and transformers will be presented in a way that the underlying idea can be understood. Corresponding medical applications in cardiovascular medicine will be named for all models, respectively. The overview is intended to show that the term artificial intelligence covers a wide range of different concepts. It should help physicians understand the principles of AI to make up one's minds about its application in cardiology. It should also enable them to evaluate results obtained with AI's help critically. [ABSTRACT FROM AUTHOR]