Search

Nowadays many research articles are prefaced with research highlights to summarize the main findings of the paper. Highlights not only help researchers precisely and quickly identify the contributions of a paper, they also enhance the discoverability of the article via search engines. We aim to automatically construct research highlights given certain segments of a research paper. We use a pointer-generator network with coverage mechanism and a contextual embedding layer at the input that encodes the input tokens into SciBERT embeddings. We test our model on a benchmark dataset, CSPubSum, and also present MixSub, a new multi-disciplinary corpus of papers for automatic research highlight generation. For both CSPubSum and MixSub, we have observed that the proposed model achieves the best performance compared to related variants and other models proposed in the literature. On the CSPubSum dataset, our model achieves the best performance when the input is only the abstract of a paper as opposed to other segments of the paper. It produces ROUGE-1, ROUGE-2 and ROUGE-L F1-scores of 38.26, 14.26 and 35.51, respectively, METEOR score of 32.62, and BERTScore F1 of 86.65 which outperform all other baselines. On the new MixSub dataset, where only the abstract is the input, our proposed model (when trained on the whole training corpus without distinguishing between the subject categories) achieves ROUGE-1, ROUGE-2 and ROUGE-L F1-scores of 31.78, 9.76 and 29.3, respectively, METEOR score of 24.00, and BERTScore F1 of 85.25.

Music generation using deep learning has received considerable attention in recent years. Researchers have developed various generative models capable of imitating musical conventions, comprehending the musical corpora, and generating new samples based on the learning outcome. Although the samples generated by these models are persuasive, they often lack musical structure and creativity. For instance, a vanilla end-to-end approach, which deals with all levels of music representation at once, does not offer human-level control and interaction during the learning process, leading to constrained results. Indeed, music creation is a recurrent process that follows some principles by a musician, where various musical features are reused or adapted. On the other hand, a musical piece adheres to a musical style, breaking down into precise concepts of timbre style, performance style, composition style, and the coherency between these aspects. Here, we study and analyze the current advances in music generation using deep learning models through different criteria. We discuss the shortcomings and limitations of these models regarding interactivity and adaptability. Finally, we draw the potential future research direction addressing multi-agent systems and reinforcement learning algorithms to alleviate these shortcomings and limitations.

Due to the increasing size of integrated circuits (ICs), their design and optimization phases (i.e., computer-aided design, CAD) grow increasingly complex. At design time, a large design space needs to be explored to find an implementation that fulfills all specifications and then optimizes metrics like energy, area, delay, reliability, etc. At run time, a large configuration space needs to be searched to find the best set of parameters (e.g., voltage/frequency) to further optimize the system. Both spaces are infeasible for exhaustive search typically leading to heuristic optimization algorithms that find some tradeoff between design quality and computational overhead. Machine learning (ML) can build powerful models that have successfully been employed in related domains. In this survey, we categorize how ML may be used and is used for design-time and run-time optimization and exploration strategies of ICs. A metastudy of published techniques unveils areas in CAD that are well explored and underexplored with ML, as well as trends in the employed ML algorithms. We present a comprehensive categorization and summary of the state of the art on ML for CAD. Finally, we summarize the remaining challenges and promising open research directions. [ABSTRACT FROM AUTHOR]

THE 19th International Workshop on Data Mining in Bioinformatics (BIOKDD 2020) was held virtually on August 24, 2020 due to the COVID-19 pandemic. BIOKDD 2020 featured the special theme of "Battling COVID-19" which particularly welcomed paper submissions and invited talks related to COVID-19 research. As a whole-day workshop, altogether 15 submissions were accepted among a total of 35 submissions, and they were divided into 4 sessions: (1) Bioinformatics, (2) Data Curation, (3) Deep Learning with Biomedical Data, and (4) Data Mining & Statistical Methods. There are also 7 invited talks by domain experts. This special section features the extended versions of 6 quality papers presented in BIOKDD 2020. [ABSTRACT FROM AUTHOR]

Tracking the trends and taking early steps accordingly is important in academia, as well as in other domains such as technology and finance. In this work, we focus on the problem of predicting the trending research topics from a collection of academic papers. Previous efforts model the problem in different ways and mostly apply classical approaches such as correlation analysis and clustering. There are also several recent neural model based solutions, however they rely on feature vectors and additional information for the trend prediction. In this work, given a collection of publications within the observation time window, we predict whether the use of a keyword will increase, decrease or be steady for the future time window (prediction window). As the solution, we propose a family of deep neural architectures that focus on generating summary representations for paper collections under the query keyword. Due to the sequence based nature of the data, Long Short-Term Memory (LSTM) module plays a core role, but it is combined with different layers in a novel way. The first group of proposed neural architectures consider each paper as a sequence of keywords and use word embeddings to construct paper collection representations. In this group, the proposed architectures differ from each other in the way year based and overall summary representations are constructed. In the second group, each paper is directly represented as a vector and the use of different paper embedding techniques are explored. The analyses of the models are performed on a variety of paper collections belonging to different academic venues, obtained from Microsoft Academic Graph data set. The experiments conducted against baseline methods show that proposed deep neural based models achieve higher trend prediction performance than the baseline models on the overall. Among the proposed models, paper embedding based models provide better results for most of the cases.

The article reports that With the continued growth of IoT devices and their deployment, manually managing and connecting them is impractical and presents multiple challenges. To that end, Zero Touch Networks that rely on software-based modules instead of dedicated propriety hardware become a viable potential solution. The overall aim of zero-touch networks is for machines to learn how to become more autonomous so that we can delegate complex, mundane tasks to them.

This special section focuses on the state-of-the-art technologies on parallel and distributed computing techniques for artificial intelligence (AI), machine learning (ML), and deep learning (DL). AI, ML, and DL can enable computers the ability to learn from a large amount of data and use the learned model to optimize a complex problem or discover rules in a complicated system. AI, ML and DL can be applied to push forward the boundaries for many domains and significantly influence our daily life. [ABSTRACT FROM AUTHOR]

Estimating depth from RGB images is a long-standing ill-posed problem, which has been explored for decades by the computer vision, graphics, and machine learning communities. Among the existing techniques, stereo matching remains one of the most widely used in the literature due to its strong connection to the human binocular system. Traditionally, stereo-based depth estimation has been addressed through matching hand-crafted features across multiple images. Despite the extensive amount of research, these traditional techniques still suffer in the presence of highly textured areas, large uniform regions, and occlusions. Motivated by their growing success in solving various 2D and 3D vision problems, deep learning for stereo-based depth estimation has attracted a growing interest from the community, with more than 150 papers published in this area between 2014 and 2019. This new generation of methods has demonstrated a significant leap in performance, enabling applications such as autonomous driving and augmented reality. In this paper, we provide a comprehensive survey of this new and continuously growing field of research, summarize the most commonly used pipelines, and discuss their benefits and limitations. In retrospect of what has been achieved so far, we also conjecture what the future may hold for deep learning-based stereo for depth estimation research. [ABSTRACT FROM AUTHOR]

The antagonistic behavior in the crowd usually exacerbates the seriousness of the situation in sudden riots, where the antagonistic emotional contagion and behavioral decision making play very important roles. However, the complex mechanism of antagonistic emotion influencing decision making, especially in the environment of sudden confrontation, has not yet been explored very clearly. In this paper, we propose an Emotional contagion-aware Deep reinforcement learning model for Antagonistic Crowd Simulation (ACSED). First, we build a group emotional contagion module based on the improved Susceptible Infected Susceptible (SIS) infection disease model, and estimate the emotional state of the group at each time step during the simulation. Then, the tendency of crowd antagonistic action is estimated based on Deep Q Network (DQN), where the agent learns the action autonomously, and leverages the mean field theory to quickly calculate the influence of other surrounding individuals on the central one. Finally, the rationality of the predicted actions by DQN is further analyzed in combination with group emotion, and the final action of the agent is determined. The proposed method in this paper is verified through several experiments with different settings. We can conclude antagonistic emotions play a critical role in the decision making of the crowd through influencing the individual behavior in the riot scenario, where individual behaviors are primarily driven by emotions and goals, rather than common rules. The experiment results also prove that the antagonistic emotion has a vital impact on the group combat, and positive emotional states are more conducive to combat. Moreover, by comparing the simulation results with real scenes, the feasibility of our method is further confirmed, which can provide good reference to formulate battle plans and improve the win rate of righteous groups in a variety of situations. [ABSTRACT FROM AUTHOR]

Surface defect is a serious problem in real-world manufacturing system and it is important to use vision-based recognition to ensure the surface quality of products. Currently, due to the ability of automatic feature extraction, deep learning models, such as convolutional neural network (CNN), have been widely used in this area. However, these CNN-based models may not solve a problem well - inter-class similarities and intra-class variations (ISIV), which affect their ability of feature extraction and thus influence their recognition performance. To address this problem, this paper introduces a graph guidance mechanism into CNN to improve the ability of feature extraction, called Graph guided Convolutional Neural Network (GCNN). Firstly, GCNN defines a graph by computing the similarities between training samples. Secondly, the graph is introduced into VGG11, a popular CNN structure, to increase the inter-class distances and decrease the intra-class distances between defect samples. Meanwhile, a learnable coefficient is introduced into the training process to balance the effect of graph guidance automatically. The experimental results on four famous surface defect datasets demonstrate that the graph guidance helps CNN models have better ability of feature extraction and thus achieve better performance. Compared with state-of-the-art models, the proposed method can achieve the best performance. Furthermore, the final discussion shows that the learnable coefficient can help the proposed model to gain better performance, and that the proposed model increases a little computation cost compared to its original CNN model. Note to Practitioners—This paper is motivated by the problem in real-world manufacturing process – inter-class similarities and intra-class variations. Most of current CNN-based models may not solve this problem well, which limits their applications in real-world system. This paper proposes a graph guidance mechanism and introduces it to the training of CNN. The proposal can improve the ability of feature extraction, and thus have better performance than those without the graph guidance. By applying the proposal to four examples, the results show that the proposal is more feasible and effective than the state-of-the-art models in surface defect recognition. Furthermore, the mechanism assists the training of CNN and is removed in the process of recognition, so it will not increase time and occupied memory in recognition, and meanwhile, only limited computation cost is increased in training process. [ABSTRACT FROM AUTHOR]

The papers in this special section focus on softwarized networking for next generation industrial cyber-physical systems (CPSs). With the emergence of embedded and ubiquitous cyberphysical applications, the rationale of blending the physical and the virtual worlds has become ever promising. These papers examine several topics that are recently concerned in the community, including the software defined architectures and implementations, advanced machine learning and data analytics solutions, blockchain-based network services and applications, network function allocation, dependable and trustable solutions, energy efficient networks and services, and other enabling technologies for integrating softwarized networks into CPSs. [ABSTRACT FROM AUTHOR]

Visual surveillance for autonomous power facility inspection is considered to bethe prominent field of study in the power industry. This research field completely focuses on either object detection or image fusion which lacks the overall consideration. By considering this, a single end-to-end object detection method by incorporating the image fusion named Fast Detection Fusion Network (FDFNet) is proposed in this paper to output qualitative fused images with detection results. The parameters in the FDFnet are greatly reduced by sharing the feature extraction network between image fusion and object detection tasks, due to which a huge reduction in computational complexity is achieved. On this basis, the object detection algorithm performance on various types of power facility images is compared and analyzed. For experimentation purposes, an IR (infrared) and VIS (visible) image acquisition system has also been designed. In addition, a dataset named CVPower with different sets of images for power facility fusion detection is constructed for this research field. Experimental results demonstrate that the proposed method can achieve the mAP of not less than 70%, process 2 frames per second, and produce high qualitative fused images. [ABSTRACT FROM AUTHOR]

A novel coronavirus disease (COVID-19) is a pandemic disease has caused 4 million deaths and more than 200 million infections worldwide (as of August 4, 2021). Rapid and accurate diagnosis of COVID-19 infection is critical to controlling the spread of the epidemic. In order to quickly and efficiently detect COVID-19 and reduce the threat of COVID-19 to human survival, we have firstly proposed a detection framework based on reinforcement learning for COVID-19 diagnosis, which constructs a mixed loss function that can integrate the advantages of multiple loss functions. This paper uses the accuracy of the validation set as the reward value, and obtains the initial model for the next epoch by searching the model corresponding to the maximum reward value in each epoch. We also have proposed a prediction framework that integrates multiple detection frameworks using parameter sharing to predict the progression of patients' disease without additional training. This paper also constructed a higher-quality version of the CT image dataset containing 247 cases screened by professional physicians, and obtained more excellent results on this dataset. Meanwhile, we used the other two COVID-19 datasets as external verifications, and still achieved a high accuracy rate without additional training. Finally, the experimental results show that our classification accuracy can reach 98.31%, and the precision, sensitivity, specificity, and AUC (Area Under Curve) are 98.82%, 97.99%, 98.67%, and 0.989, respectively. The accuracy of external verification can reach 93.34% and 91.05%. What's more, the accuracy of our prediction framework is 91.54%. A large number of experiments demonstrate that our proposed method is effective and robust for COVID-19 detection and prediction. [ABSTRACT FROM AUTHOR]

Multi-focus image fusion (MFIF) is an important area in image processing. Since 2017, deep learning has been introduced to the field of MFIF and various methods have been proposed. However, there is a lack of survey papers that discuss deep learning-based MFIF methods in detail. In this study, we fill this gap by giving a detailed survey on deep learning-based MFIF algorithms, including methods, datasets and evaluation metrics. To the best of our knowledge, this is the first survey paper that focuses on deep learning-based approaches in the field of MFIF. Besides, extensive experiments have been conducted to compare the performance of deep learning-based MFIF algorithms with conventional MFIF approaches. By analyzing qualitative and quantitative results, we give some observations on the current status of MFIF and discuss some future prospects of this field. [ABSTRACT FROM AUTHOR]

In this paper, considering spatiotemporal correlations, we propose a novel short-term traffic flow prediction method that is based on diverse ensemble deep learning. First, a new measurement function of spatial correlation is established, quantifying the spatial correlation of traffic flow parameters. Second, the convolutional neural network model embedded long short-term memory (LSTM-CNN) is used, enhancing the learning ability of the spatiotemporal correlation of traffic flow parameters. Third, according to the network structure, ensemble rules of diverse LSTM-CNNs are constructed, combining a series of different and moderately accurate LSTM-CNN models and improving the robustness of the algorithm. Finally, a dynamic optimization method for the weight parameters of the ensemble elements is proposed to accommodate the changes in the actual road networks. The experiments show that the measuring method of spatial correlations, the ensemble combination rules of diverse LSTM-CNN and the dynamic optimization method of ensemble element weight parameters, which are proposed in this paper, have positive effects on improving the performance of the prediction algorithm. In addition, the short-term traffic flow prediction method proposed in this paper can adapt to changes in the traffic flow. It can obtain a better prediction effect even with a small sample, and the prediction performance is better than that of the other four classical short-term traffic flow prediction methods. [ABSTRACT FROM AUTHOR]

Heart sound auscultation has been applied in clinical usage for early screening of cardiovascular diseases. Due to the high demand for auscultation expertise, automatic auscultation can help with auxiliary diagnosis and reduce the burden of training professional clinicians. Nevertheless, there is a limit to classic machine learning's performance improvement in the era of Big Data. Deep learning has outperformed classic machine learning in many research fields, as it employs more complex model architectures with a stronger capability of extracting effective representations. Moreover, it has been successfully applied to heart sound analysis in the past years. As most review works about heart sound analysis were carried out before 2017, the present survey is the first to work on a comprehensive overview to summarise papers on heart sound analysis with deep learning published in 2017–2022. This work introduces both classic machine learning and deep learning for comparison, and further offer insights about the advances and future research directions in deep learning for heart sound analysis. [ABSTRACT FROM AUTHOR]

In this paper, we present a comprehensive review of the imbalance problems in object detection. To analyze the problems in a systematic manner, we introduce a problem-based taxonomy. Following this taxonomy, we discuss each problem in depth and present a unifying yet critical perspective on the solutions in the literature. In addition, we identify major open issues regarding the existing imbalance problems as well as imbalance problems that have not been discussed before. Moreover, in order to keep our review up to date, we provide an accompanying webpage which catalogs papers addressing imbalance problems, according to our problem-based taxonomy. Researchers can track newer studies on this webpage available at: https://github.com/kemaloksuz/ObjectDetectionImbalance. [ABSTRACT FROM AUTHOR]

This paper investigates a new class of carrier-sense multiple access (CSMA) protocols that employ deep reinforcement learning (DRL) techniques, referred to as carrier-sense deep-reinforcement learning multiple access (CS-DLMA). The goal of CS-DLMA is to enable efficient and equitable spectrum sharing among a group of co-located heterogeneous wireless networks. Existing CSMA protocols, such as the medium access control (MAC) protocol of WiFi, are designed for a homogeneous network in which all nodes adopt the same protocol. Such protocols suffer from severe performance degradation in a heterogeneous environment where there are nodes adopting other MAC protocols. CS-DLMA aims to circumvent this problem by making use of DRL. In particular, this paper adopts α-fairness as the general objective of CS-DLMA. With α-fairness, CS-DLMA can achieve a range of different objectives (e.g., maximizing sum throughput, achieving proportional fairness, or achieving max-min fairness) when coexisting with other MACs by changing the value of α. A salient feature of CS-DLMA is that it can achieve these objectives without knowing the coexisting MACs through a learning process based on DRL. The underpinning DRL technique in CS-DLMA is deep Q-network (DQN). However, the conventional DQN algorithms are not suitable for CS-DLMA due to their uniform time-step assumption. In CSMA protocols, time steps are non-uniform in that the time duration required for carrier sensing is smaller than the duration of data transmission. This paper introduces a non-uniform time-step formulation of DQN to address this issue. Our simulation results show that CS-DLMA can achieve the general α-fairness objective when coexisting with TDMA, ALOHA, and WiFi protocols by adjusting its own transmission strategy. Interestingly, we also find that CS-DLMA is more Pareto efficient than other CSMA protocols, e.g., p-persistent CSMA, when coexisting with WiFi. Although this paper focuses on the use of our non-uniform time-step DQN formulation in wireless networking, we believe this new DQN formulation can also find use in other domains. [ABSTRACT FROM AUTHOR]

Unmanned Aerial Vehicle (UAV) inspection has gradually replaced manual inspection of transmission tower, which produces many images. While it is laborious and time-consuming to manually analyze these images, there are also challenges in automatically detecting small-size defective components such as bolts in transmission tower images, due to problems including complex background, small size, and many similar objects of bolts. In this paper, by virtue of multi-scale features and context information, we propose a deep neural network named Camp-Net (Context Information and Multi-Scale Pyramid Network) to identify bolts defect in transmission tower images. First, multi-scale feature fusion combines deep features and shallow features in convolutional networks to detect small-size bolts. Second, context information fusion puts the information around bolts into the detection network to remove the disturbance of complex background and similar objects. An image dataset containing defective bolts and normal bolts is constructed for model training and testing. Experimental results show that bolts with loose pins and bolts without pins among fittings in transmission tower can be accurately identified with the proposed model. The Average Precision (AP) of defective bolts detection of this model can be 11.4% higher than that of the commonly used high performance model, Faster R-CNN. [ABSTRACT FROM AUTHOR]

The fifth generation New Radio vehicle-to-everything (5G NR-V2X) has higher requirements for delay and reliability, which brings challenges to the physical layer signal processing. Mobile edge computing (MEC) can provide larger capacity data storage and more efficient computation through localization of on-board unit (OBU) and next generation Node B (gNB) services. This paper combines the channel estimation of New Radio vehicle-to-infrastructure (NR-V2I) communication baseband signal processing with MEC and designs an intelligent channel estimation framework. In the MEC server, this paper proposes a channel estimation algorithm based on deep learning. This algorithm uses a one-dimensional convolutional neural network (1D CNN) to complete frequency-domain interpolation and conditional recurrent unit (CRU) for time-domain state prediction. Additional velocity coding vector and multipath coding vector track changes in the environment, and accurately train channel data in different mobile environments. System simulation and analysis show that the proposed algorithm improves the channel estimation accuracy, reduces the bit error rate, and enhances the robustness compared with the representative channel estimation algorithms for NR-V2I. [ABSTRACT FROM AUTHOR]

The explosive growth of Internet-of-Things (IoT) applications such as smart cities and Industry 4.0 have led to drastic increase in demand for wireless bandwidth, hence motivating the rapid development of new techniques for enhancing spectrum utilization needed by new generation wireless communication technologies. Among others, dynamic spectrum access (DSA) is one of the most widely accepted approaches. In this paper, as an enhancement of existing works, we take into consideration of inter-node collaborations in a dynamic spectrum environment. Typically, in such distributed circumstances, intelligent dynamic spectrum access almost invariably relies on self-learning to achieve dynamic spectrum access improvement. Whereas, this paper proposes a DSA scheme based on deep reinforcement learning to enhance spectrum and access efficiency. Unlike traditional Q-learning-based DSA, we introduce the following to enhance the spectrum efficiency in dynamic IoT spectrum environments. First, deep double Q-learning is adopted to perform local self-spectrum-learning for IoT terminals in order to achieve better dynamic access accuracy. Second, to accelerate learning convergence, federated learning (FL) in edge nodes is used to improve the self-learning. Third, multiple secondary users, who do not interfere with each other and have similar operation condition, are clustered for federated learning to enhance the efficiency of deep reinforcement learning. Comparing with the traditional distributed DSA with deep learning, the proposed scheme has faster access convergence speed due to the characteristic of global optimization for federated learning. Based on this, a framework of federated deep reinforcement learning (FDRL) for DSA is proposed. Furthermore, this scheme preserves privacy of IoT users in that FDRL only requires model parameters to be uploaded to edge servers. Simulations are performed to show the effectiveness of theproposed FDRL-based DSA framework. [ABSTRACT FROM AUTHOR]

Since gait can be utilized to identify individuals from a far distance without their interaction and coordination, recently many gait recognition methods have been proposed. However, due to a real-world scenario of clothing changes, a degradation occurs for most of these methods. Thus in this paper, a more efficient gait recognition method is proposed to address the problem of clothing variances. First, part-based gait features are formulated from two different perspectives, i.e., the separated body parts that are more robust to clothing changes and the estimated human skeleton key-point regions. It is reasonable to formulate such features for cloth-changing gait recognition, because these two perspectives are both less vulnerable to clothing changes. Given that each feature has its own advantages and disadvantages, a more efficient gait feature is generated in this paper by assembling these two features together. Moreover, since local features are more discriminative than global features, in this paper more attention is focused on the local short-range features. Also, unlike most methods, in our method we treat the estimated key-point features as a set of word embeddings, and a transformer encoder is specifically used to learn the dependence of each correlative key-points. The robustness and effectiveness of our proposed method are certified by experiments on CASIA Gait Dataset B, and it has achieved the state-of-the-art performance on this dataset. [ABSTRACT FROM AUTHOR]

The IEEE Transactions on Semiconductor Manufacturing congratulates Haoyu Yang, Shuhe Li, Wen Chen, Piyush Pathak, Frank Gennari, Ya-Chieh Lai, and Bei Yu whose paper DeePattern: Layout Pattern Generation With Transforming Convolutional Auto-Encoder (Vol. 35, No.1, February 2022) was selected as the Best Paper for 2022 by a team of Associate Editors. This paper proposes generative machine learning models to synthesize VLSI layouts through a pattern generation framework that reduces the challenging creation problem into two simpler subproblems with the aid of an efficient squish pattern representation. The problem is treated by designing an innovative transforming convolutional auto-encoder in TCAE architecture, aiming to generate efficient and representative pattern shapes. The paper introduces a GAN-guided TCAE ML analysis that targets massive DRC-clean and content-specific pattern generation following certain design rules, without losing pattern library diversity. Through experiments on 7nm EUV designs, the authors demonstrate that each latent vector node in TCAE has a real physical meanings in layout domain and transformations on latent vectors can produce additional topologies of interest. Results show that the generated pattern library exhibits larger pattern number and pattern diversity compared to the traditional state-of-the-art industry layout generator. [ABSTRACT FROM AUTHOR]

This paper proposes a deep learning framework to design distributed compression strategies in which distributed agents need to compress high-dimensional observations of a source, then send the compressed bits via bandwidth limited links to a fusion center for source reconstruction. Further, we require the compression strategy to be progressive so that it can adapt to the varying link bandwidths between the agents and the fusion center. Moreover, to ensure scalability, we investigate strategies that depend only on the local channel state information (CSI) at each agent. Toward this end, we use a data-driven approach in which the progressive linear combination and uniform quantization strategy at each agent are trained as a function of its local CSI. To deal with the challenges of modeling the quantization operations (which always produce zero gradients in the training of neural networks), we propose a novel approach of exploiting the statistics of the batch training data to set the dynamic ranges of the uniform quantizers. Numerically, we show that the proposed distributed estimation strategy designed with only local CSI can significantly reduce the signaling overhead and can achieve a lower mean-squared error distortion for source reconstruction than state-of-the-art designs that require global CSI at comparable overall communication cost. [ABSTRACT FROM AUTHOR]

In this paper, we propose to use commercial off-the-shelf (COTS) monostatic RFID devices (i.e. which use a single antenna at a time for both transmitting and receiving RFID signals to and from the tags) to monitor browsing activity of customers in front of display items in places such as retail stores. To this end, we propose TagSee, a multi-person imaging system based on monostatic RFID imaging. TagSee is based on the insight that when customers are browsing the items on a shelf, they stand between the tags deployed along the boundaries of the shelf and the reader, which changes the multi-paths that the RFID signals travel along, and both the RSS and phase values of the RFID signals that the reader receives change. Based on these variations observed by the reader, TagSee constructs a coarse grained image of the customers. Afterwards, TagSee identifies the items that are being browsed by the customers by analyzing the constructed images. The key novelty of this paper is on achieving browsing behavior monitoring of multiple customers in front of display items by constructing coarse grained images via robust, analytical model-driven deep learning based, RFID imaging. To achieve this, we first mathematically formulate the problem of imaging humans using monostatic RFID devices and derive an approximate analytical imaging model that correlates the variations caused by human obstructions in the RFID signals. Based on this model, we then develop a deep learning framework to robustly image customers with high accuracy. We implement TagSee scheme using a Impinj Speedway R420 reader and SMARTRAC DogBone RFID tags. TagSee can achieve a TPR of more than ${\sim }90\%$ ∼ 90 % and a FPR of less than ${\sim }10\%$ ∼ 10 % in multi-person scenarios using training data from just 3-4 users. [ABSTRACT FROM AUTHOR]

Magnetic Resonance Imaging (MRI) has commonly been used to detect and diagnose brain disease and monitor treatment as non-invasive imaging technology. MRI produces three-dimensional images that help neurologists to identify anomalies from brain images precisely. However, this is a time-consuming and labor-intensive process. The improvement in machine learning and efficient computation provides a computer-aid solution to analyze MRI images and identify the abnormality quickly and accurately. Image segmentation has become a hot and research-oriented area in the medical image analysis community. The computer-aid system for brain abnormalities identification provides the possibility for quickly classifying the disease for early treatment. This article presents a review of the research papers (from 1998 to 2020) on brain tumors segmentation from MRI images. We examined the core segmentation algorithms of each research paper in detail. This article provides readers with a complete overview of the topic and new dimensions of how numerous machine learning and image segmentation approaches are applied to identify brain tumors. By comparing the state-of-the-art and new cutting-edge methods, the deep learning methods are more effective for the segmentation of the tumor from MRI images of the brain. [ABSTRACT FROM AUTHOR]

Retinal image registration is a critical task in the diagnosis and treatment of various eye diseases. And as a relatively new imaging method, optical coherence tomography (OCT) has been widely used in the diagnosis of retinal diseases. This paper is devoted to retinal OCT image registration methods and their clinical applications. Registration methods including volumetric transformation-based registration methods and image features-based registration methods are systematically reviewed. Furthermore, to better understanding these methods, their applications in correcting scanning artifacts, reducing speckle noise, fusing and splicing images and evaluating longitudinal disease progression are studied as well. At the end of this paper, registration of retina with serious pathology and registration with deep learning technique are also discussed. [ABSTRACT FROM AUTHOR]

Bi-Level Optimization (BLO) is originated from the area of economic game theory and then introduced into the optimization community. BLO is able to handle problems with a hierarchical structure, involving two levels of optimization tasks, where one task is nested inside the other. In machine learning and computer vision fields, despite the different motivations and mechanisms, a lot of complex problems, such as hyper-parameter optimization, multi-task and meta learning, neural architecture search, adversarial learning and deep reinforcement learning, actually all contain a series of closely related subproblms. In this paper, we first uniformly express these complex learning and vision problems from the perspective of BLO. Then we construct a best-response-based single-level reformulation and establish a unified algorithmic framework to understand and formulate mainstream gradient-based BLO methodologies, covering aspects ranging from fundamental automatic differentiation schemes to various accelerations, simplifications, extensions and their convergence and complexity properties. Last but not least, we discuss the potentials of our unified BLO framework for designing new algorithms and point out some promising directions for future research. A list of important papers discussed in this survey, corresponding codes, and additional resources on BLOs are publicly available at: https://github.com/vis-opt-group/BLO. [ABSTRACT FROM AUTHOR]

Deep learning models have been shown to be vulnerable to adversarial attacks. Adversarial attacks are imperceptible perturbations added to an image such that the deep learning model misclassifies the image with a high confidence. Existing adversarial defenses validate their performance using only the classification accuracy. However, classification accuracy by itself is not a reliable metric to determine if the resulting image is “adversarial-free”. This is a foundational problem for online image recognition applications where the ground-truth of the incoming image is not known and hence we cannot compute the accuracy of the classifier or validate if the image is “adversarial-free” or not. This paper proposes a novel privacy preserving framework for defending Black box classifiers from adversarial attacks using an ensemble of iterative adversarial image purifiers whose performance is continuously validated in a loop using Bayesian uncertainties. The proposed approach can convert a single-step black box adversarial defense into an iterative defense and proposes three novel privacy preserving Knowledge Distillation (KD) approaches that use prior meta-information from various datasets to mimic the performance of the Black box classifier. Additionally, this paper proves the existence of an optimal distribution for the purified images that can reach a theoretical lower bound, beyond which the image can no longer be purified. Experimental results on six public benchmark datasets namely: 1) Fashion-MNIST, 2) CIFAR-10, 3) GTSRB, 4) MIO-TCD, 5) Tiny-ImageNet, and 6) MS-Celeb show that the proposed approach can consistently detect adversarial examples and purify or reject them against a variety of adversarial attacks. [ABSTRACT FROM AUTHOR]

Entity resolution (ER) identifies data instances referring to the same real-world entity and has received enormous research attention. In this paper, we examine the task of ER from a broader perspective, with its input extended from textual records, which are conventionally studied in the literature, to other modalities such as check-in sequences, GPS trajectories and surveillance video frames to generate new applications. Our goal in this paper is to design an effective model to uniformly support all these ER applications with different input formats. Technically, we fully exploit the semantic contexts of embedding vectors for the pair of input instances. In particular, we propose an integrated multi-context attention framework that takes into account self-attention, pair-attention and global-attention from three types of context. The idea can be further extended to incorporate attribute attention in order to support structured datasets. We conduct extensive experiments on a diverse class of entity resolutions tasks, including tasks on unstructured, structured and dirty textual records, check-in sequences, GPS trajectories and surveillance video frames. The experimental results verified the effectiveness and generality of our model. When compared with strong baselines in these applications, our model can achieve superior or comparative performance. [ABSTRACT FROM AUTHOR]

Deep learning methods have advanced quickly in brain imaging analysis over the past few years, but they are usually restricted by the limited labeled data. Pre-trained model on unlabeled data has presented promising improvement in feature learning in many domains, such as natural language processing. However, this technique is under-explored in brain network analysis. In this paper, we focused on pre-training methods with Transformer networks to leverage existing unlabeled data for brain functional network classification. First, we proposed a Transformer-based neural network, named as BrainNPT, for brain functional network classification. The proposed method leveraged token as a classification embedding vector for the Transformer model to effectively capture the representation of brain networks. Second, we proposed a pre-training framework for BrainNPT model to leverage unlabeled brain network data to learn the structure information of brain functional networks. The results of classification experiments demonstrated the BrainNPT model without pre-training achieved the best performance with the state-of-the-art models, and the BrainNPT model with pre-training strongly outperformed the state-of-the-art models. The pre-training BrainNPT model improved 8.75% of accuracy compared with the model without pre-training. We further compared the pre-training strategies and the data augmentation methods, analyzed the influence of the parameters of the model, and explained the trained model.

Early diagnosis and intervention of depression promote complete recovery, with its traditional clinical assessments depending on the diagnostic scales, clinical experience of doctors and patient cooperation. Recent researches indicate that functional near-infrared spectroscopy (fNIRS) based on deep learning provides a promising approach to depression diagnosis. However, collecting large fNIRS datasets within a standard experimental paradigm remains challenging, limiting the applications of deep networks that require more data. To address these challenges, in this paper, we propose an fNIRS-driven depression recognition architecture based on cross-modal data augmentation (fCMDA), which converts fNIRS data into pseudo-sequence activation images. The approach incorporates a time-domain augmentation mechanism, including time warping and time masking, to generate diverse data. Additionally, we design a stimulation task-driven data pseudo-sequence method to map fNIRS data into pseudo-sequence activation images, facilitating the extraction of spatial-temporal, contextual and dynamic characteristics. Ultimately, we construct a depression recognition model based on deep classification networks using the imbalance loss function. Extensive experiments are performed on the two-class depression diagnosis and five-class depression severity recognition, which reveal impressive results with accuracy of 0.905 and 0.889, respectively. The fCMDA architecture provides a novel solution for effective depression recognition with limited data.

Brain-computer interfaces (BCIs) can enable direct communication with assistive devices by recording and decoding signals from the brain. To achieve high performance, many electrodes will be used, such as the recently developed invasive BCIs with channel numbers up to hundreds or even thousands. For those high-throughput BCIs, channel selection is important to reduce signal redundancy and invasiveness while maintaining decoding performance. However, such endeavour is rarely reported for invasive BCIs, especially those using deep learning methods. Two deep learning-based methods, referred to as Gumbel and STG, were proposed in this paper. They were evaluated using the Stereo-electroencephalography (SEEG) signals, and compared with three other methods, including manual selection, mutual information-based method (MI), and all channels (all channels without selection). The task is to classify the SEEG signals into five movements using channels selected by each method. When 10 channels were selected, the mean classification accuracies using Gumbel, STG (referred to as STG-10), manual selection, and MI selection were 65%, 60%, 60%, and 47%, respectively, whilst the accuracy was 59% using all channels (no selection). In addition, an investigation of the selected channels showed that Gumbel and STG have successfully identified the pre-central and post-central areas, which are closely related to motor control. Both Gumbel and STG successfully selected the informative channels in SEEG recordings while maintaining decoding accuracy. This study enables future high-throughput BCIs using deep learning methods, to identify useful channels and reduce computing and wireless transmission pressure.

The olfactory system enables humans to smell different odors, which are closely related to emotions. The high temporal resolution and non-invasiveness of Electroencephalogram (EEG) make it suitable to objectively study human preferences for odors. Effectively learning the temporal dynamics and spatial information from EEG is crucial for detecting odor-induced emotional valence. In this paper, we propose a deep learning architecture called Temporal Attention with Spatial Autoencoder Network (TASA) for predicting odor-induced emotions using EEG. TASA consists of a filter-bank layer, a spatial encoder, a time segmentation layer, a Long Short-Term Memory (LSTM) module, a multi-head self-attention (MSA) layer, and a fully connected layer. We improve upon the previous work by utilizing a two-phase learning framework, using the autoencoder module to learn the spatial information among electrodes by reconstructing the given input with a latent representation in the spatial dimension, which aims to minimize information loss compared to spatial filtering with CNN. The second improvement is inspired by the continuous nature of the olfactory process; we propose to use LSTM-MSA in TASA to capture its temporal dynamics by learning the intercorrelation among the time segments of the EEG. TASA is evaluated on an existing olfactory EEG dataset and compared with several existing deep learning architectures to demonstrate its effectiveness in predicting olfactory-triggered emotional responses. Interpretability analyses with DeepLIFT also suggest that TASA learns spatial-spectral features that are relevant to olfactory-induced emotion recognition.

Automatic identification of visual learning style in real time using raw electroencephalogram (EEG) is challenging. In this work, inspired by the powerful abilities of deep learning techniques, deep learning-based models are proposed to learn high-level feature representation for EEG visual learning identification. Existing computer-aided systems that use electroencephalograms and machine learning can reasonably assess learning styles. Despite their potential, offline processing is often necessary to eliminate artifacts and extract features, making these methods unsuitable for real-time applications. The dataset was chosen with 34 healthy subjects to measure their EEG signals during resting states (eyes open and eyes closed) and while performing learning tasks. The subjects displayed no prior knowledge of the animated educational content presented in video format. The paper presents an analysis of EEG signals measured during a resting state with closed eyes using three deep learning techniques: Long-term, short-term memory (LSTM), Long-term, short-term memory-convolutional neural network (LSTM-CNN), and Long-term, short-term memory-Fully convolutional neural network (LSTM-FCNN). The chosen techniques were based on their suitability for real-time applications with varying data lengths and the need for less computational time. The optimization of hypertuning parameters has enabled the identification of visual learners through the implementation of three techniques. LSTM-CNN technique has the highest average accuracy of 94%, a sensitivity of 80%, a specificity of 92%, and an F1 score of 94% when identifying the visual learning style of the student out of all three techniques. This research has shown that the most effective method is the deep learning-based LSTM-CNN technique, which accurately identifies a student's visual learning style.

Sleep staging serves as a fundamental assessment for sleep quality measurement and sleep disorder diagnosis. Although current deep learning approaches have successfully integrated multimodal sleep signals, enhancing the accuracy of automatic sleep staging, certain challenges remain, as follows: 1) optimizing the utilization of multi-modal information complementarity, 2) effectively extracting both long- and short-range temporal features of sleep information, and 3) addressing the class imbalance problem in sleep data. To address these challenges, this paper proposes a two-stream encode-decoder network, named TSEDSleepNet, which is inspired by the depth sensitive attention and automatic multi-modal fusion (DSA2F) framework. In TSEDSleepNet, a two-stream encoder is used to extract the multiscale features of electrooculogram (EOG) and electroencephalogram (EEG) signals. And a self-attention mechanism is utilized to fuse the multiscale features, generating multi-modal saliency features. Subsequently, the coarser-scale construction module (CSCM) is adopted to extract and construct multi-resolution features from the multiscale features and the salient features. Thereafter, a Transformer module is applied to capture both long- and short-range temporal features from the multi-resolution features. Finally, the long- and short-range temporal features are restored with low-layer details and mapped to the predicted classification results. Additionally, the Lovász loss function is applied to alleviate the class imbalance problem in sleep datasets. Our proposed method was tested on the Sleep-EDF-39 and Sleep-EDF-153 datasets, and it achieved classification accuracies of 88.9% and 85.2% and Macro-F1 scores of 84.8% and 79.7%, respectively, thus outperforming conventional traditional baseline models. These results highlight the efficacy of the proposed method in fusing multi-modal information. This method has potential for application as an adjunct tool for diagnosing sleep disorders.

Computational biomechanical analysis plays a pivotal role in understanding and improving human movements and physical functions. Although physics-based modeling methods can interpret the dynamic interaction between the neural drive to muscle dynamics and joint kinematics, they suffer from high computational latency. In recent years, data-driven methods have emerged as a promising alternative due to their fast execution speed, but label information is still required during training, which is not easy to acquire in practice. To tackle these issues, this paper presents a novel physics-informed deep learning method to predict muscle forces without any label information during model training. In addition, the proposed method could also identify personalized muscle-tendon parameters. To achieve this, the Hill muscle model-based forward dynamics is embedded into the deep neural network as the additional loss to further regulate the behavior of the deep neural network. Experimental validations on the wrist joint from six healthy subjects are performed, and a fully connected neural network (FNN) is selected to implement the proposed method. The predicted results of muscle forces show comparable or even lower root mean square error (RMSE) and higher coefficient of determination compared with baseline methods, which have to use the labeled surface electromyography (sEMG) signals, and it can also identify muscle-tendon parameters accurately, demonstrating the effectiveness of the proposed physics-informed deep learning method.

Major Depression Disorder (MDD) is a common yet destructive mental disorder that affects millions of people worldwide. Making early and accurate diagnosis of it is very meaningful. Recently, EEG, a non-invasive technique of recording spontaneous electrical activity of brains, has been widely used for MDD diagnosis. However, there are still some challenges in data quality and data size of EEG: (1) A large amount of noise is inevitable during EEG collection, making it difficult to extract discriminative features from raw EEG; (2) It is difficult to recruit a large number of subjects to collect sufficient and diverse data for model training. Both of the challenges cause the overfitting problem, especially for deep learning methods. In this paper, we propose DiffMDD, a diffusion-based deep learning framework for MDD diagnosis using EEG. Specifically, we extract more noise-irrelevant features to improve the model's robustness by designing the Forward Diffusion Noisy Training Module. Then we increase the size and diversity of data to help the model learn more generalized features by designing the Reverse Diffusion Data Augmentation Module. Finally, we re-train the classifier on the augmented dataset for MDD diagnosis. We conducted comprehensive experiments to test the overall performance and each module's effectiveness. The framework was validated on two public MDD diagnosis datasets, achieving the state-of-the-art performance.

This paper proposes a novel scalable reinforcement learning approach for simultaneous routing and spectrum access in wireless ad-hoc networks. In most previous works on reinforcement learning for network optimization, the network topology is assumed to be fixed, and a different agent is trained for each transmission node—this limits scalability and generalizability. Further, routing and spectrum access are typically treated as separate tasks. Moreover, the optimization objective is usually a cumulative metric along the route, e.g., number of hops or delay. In this paper, we account for the physical-layer signal-to-interference-plus-noise ratio (SINR) in a wireless network and further show that bottleneck objective such as the minimum SINR along the route can also be optimized effectively using reinforcement learning. Specifically, we propose a scalable approach in which a single agent is associated with each flow and makes routing and spectrum access decisions as it moves along the frontier nodes. The agent is trained according to the physical-layer characteristics of the environment using a novel rewarding scheme based on the Monte Carlo estimation of the future bottleneck SINR. It learns to avoid interference by intelligently making joint routing and spectrum allocation decisions based on the geographical location information of the neighbouring nodes. [ABSTRACT FROM AUTHOR]

The document is the table of contents for the May 2024 issue of the IEEE Transactions on Semiconductor Manufacturing. It includes papers on various topics such as yield modeling, analysis, and enhancement, advanced processing, advanced process control, equipment and automation technology, non-silicon materials, emerging areas, and environment, safety, and health. The articles cover subjects such as chip-scale chemical mechanical polishing, threshold voltage deviation monitoring, curvilinear standard cell design, fabrication of silicon nanocone arrays, improving the reliability of through silicon vias, energy consumption and carbon emission reduction in HVAC systems, low-k silicon dioxide synthesis, chemical mechanical polishing of diamond epitaxial layers, conditional variable selection based on deep learning, and eco-friendly dry-cleaning and diagnostics of silicon dioxide deposition chambers. The document also includes an announcement for the call for nominations for the 2024 IEEE EDS Early Career Award. [Extracted from the article]

Digital watermarking technology and watermark attack methods are mutually reinforcing and complementary. Currently, traditional watermark attack methods are relatively mature, but these traditional attack methods will inevitably damage the visual quality of original images (OIs). Therefore, this paper proposes a covert attack method called residual dense based imperceptible watermark attack network (RD-IWAN). First, this paper designs a watermark attack residual dense network (WARDN) based on the residual dense network (RDN), which can effectively remove the watermark information in the middle and low frequency features of the watermarked image (WMI). Second, to improve the attack ability of the network, this paper innovatively proposes a progressive preprocessing method based on the information enhancement preprocessing method. Concurrently, to ensure the imperceptibility of this watermark attack method, a comprehensive loss function that combines the perceptual loss and mean square error loss (MSE) of OI and attacked watermarked image (AWMI) is designed in this study. Finally, attack experiments are designed and performed on watermarks with different embedding strengths and sizes. Experimental results show that, compared to traditional attack methods, the watermark attack method proposed in this paper exhibits stronger attack ability and higher imperceptibility. [ABSTRACT FROM AUTHOR]

This paper introduces video domain generalization where most video classification networks degenerate due to the lack of exposure to the target domains of divergent distributions. We observe that the global temporal features are less generalizable, due to the temporal domain shift that videos from other unseen domains may have an unexpected absence or misalignment of the temporal relations. This finding has motivated us to solve video domain generalization by effectively learning the local-relation features of different timescales that are more generalizable, and exploiting them along with the global-relation features to maintain the discriminability. This paper presents the VideoDG framework with two technical contributions. The first is a new deep architecture named the Adversarial Pyramid Network, which improves the generalizability of video features by capturing the local-relation, global-relation, and cross-relation features progressively. On the basis of pyramid features, the second contribution is a new and robust approach of adversarial data augmentation that can bridge different video domains by improving the diversity and quality of augmented data. We construct three video domain generalization benchmarks in which domains are divided according to different datasets, different consequences of actions, or different camera views, respectively. VideoDG consistently outperforms the combinations of previous video classification models and existing domain generalization methods on all benchmarks. [ABSTRACT FROM AUTHOR]

The future of autonomous driving is slowly approaching, but there are still many steps to take before it can become a reality. It is crucial to pay attention to road infrastructure, because without it, intelligent vehicles will not be able to operate reliably, and it will never be possible to dispense of driver’s control. This paper presents the work carried out for the detection of road markings damage using computer vision techniques. This is a complex task for which there are currently not many papers and large image sets in the literature. This study uses images from the public Road Damage Detection dataset for the D44 defect and also provides 971 new labelled images for Spanish roads. For this purpose, three detectors based on deep learning architectures (Faster RCNN, SDD and EfficientDet) have been used and single-source and mixed-source models have been studied to find the model that best fits the target images. Finally, F1-score values reaching 0.929 and 0.934 have been obtained for Japanese and Spanish images respectively which improve the state-of-the-art results by 25%. It can be concluded that the results of this study are promising, although the collection of many more images will be necessary for the scientific community to continue advancing in the future in this field of research. [ABSTRACT FROM AUTHOR]

The research on loop closure detection has been carried out for the last many years; however, loop closure detection efficiency is still not that good and has been affected by many factors, including illumination conditions, weather conditions, seasons, and viewpoint changes. The research on loop closure detection from a different viewpoint is still an open research problem. The paper proposes an efficient solution to loop closure detection from different viewpoints by using landmarks instead of whole frames and taking the deep and robust features of deep learning instead of handcrafted features. A different kind of training approach is used to train the deep CNN to get highly abstract embeddings of input landmarks. The approach has many advantages over the traditional training approach and can solve many complex problems. This paper has used this approach to force similar or closer embeddings for similar landmarks and is forced to have a large gap in embeddings of different landmarks. The proposed method endeavors viewpoint invariant features, and the astonishing power of deep learning makes the features robust to viewpoint changes, occlusions, and illumination variations. The proposed visual SLAM system is tested on six publicly available datasets, and the results are compared with the most popular Bag of Words methods like DBoW2, DBoW3, and state-of-the-art deep learning methods AlexNet, ResNeXt, FlyNet, AFDPR and Siamese network. The results show that our method is efficient in finding loop closures candidates from different viewpoints. Code is available at https://github.com/IRMVLab/Step-wise-Learning. [ABSTRACT FROM AUTHOR]

The problem under consideration in this paper is how to avoid deadlocks and livelocks in the paradigm of Petri nets (PNs). Although deadlock and livelock avoidance has been extensively studied in existing literature, fewer results can be applied to practical automated manufacturing systems (AMSs) due to expensive computations and reduced permissiveness. In this paper, we propose an efficient and maximally permissive control scheme to avoid deadlocks and livelocks by using critical distance. From the perspective of the reachability graph of a PN, critical distance is equal to the maximum number of transitions among all transition sequences from any critical state to its corresponding deadlock or livelock state. First, we show how to calculate the critical distance of a PN in an efficient way by using a portion of reachable states. Then, local reachability graphs are established based on critical distance to avoid deadlocks and livelocks. The provided policy is maximally permissive since only the necessary minimum number of illegal transitions are forbidden at each state. Several representative examples are presented to illustrate our approach. Note to Practitioners—Deadlock and livelock avoidance is a crucial problem in automated manufacturing systems (AMSs). Various methods have been proposed to deal with deadlocks and livelocks by researchers and practitioners. However, there exist fewer works that focus on deep exploration regarding how long a deadlock or livelock will occur from a safe state. As a consequence, the existing works suffer from the disadvantages of being too aggressive or conservative, i.e., checking the whole state space or forbidding many acceptable states when avoiding deadlocks and livelocks. Through the analysis of a partial of reachable states, this paper derives the critical distance of Petri nets modeling AMSs, which is equal to the maximum number of transitions among all transition sequences from any critical marking to its corresponding deadlock or livelock marking. All deadlocks and livelocks, if exist, can be detected at any given marking within the critical distance. Our approach not only is maximally permissive but also explores only a partial of states, thereby mitigating state explosion problem. [ABSTRACT FROM AUTHOR]

To accurately pour drinks into various containers is an essential skill for service robots. However, drink pouring is a dynamic process and difficult to model. Traditional deep imitation learning techniques for implementing autonomous robotic pouring have an inherent black-box effect and require a large amount of demonstration data for model training. To address these issues, an Explainable Hierarchical Imitation Learning (EHIL) method is proposed in this paper such that a robot can learn high-level general knowledge and execute low-level actions across multiple drink pouring scenarios. Moreover, with the EHIL method, a logical graph can be constructed for task execution, through which the decision-making process for action generation can be made explainable to users and the causes of failure can be traced out. Based on the logical graph, the framework is manipulable to achieve different targets while the adaptability to unseen scenarios can be achieved in an explainable manner. A series of experiments have been conducted to verify the effectiveness of the proposed method. Results indicate that EHIL outperforms the traditional behavior cloning method in terms of success rate, adaptability, manipulability, and explainability. Note to Practitioners—Pouring liquids is a common activity in people’s daily lives and all wet-lab industries. Drink pouring dynamic control is difficult to model, while the accurate perception of flow is challenging. To enable the robot to learn under unknown dynamics via observing the human demonstration, deep imitation learning can be used. To address the limitations of traditional deep neural networks, an Explainable Hierarchical Imitation Learning (EHIL) method is proposed in this paper. The proposed method enables the robot to learn a sequence of reasonable pouring phases for performing the task rather than simply execute the task via traditional behavior cloning. In this way, explainability and safety can be ensured. Manipulability can be achieved by reconstructing the logical graph. The target of this research is to obtain pouring dynamics via the learning method and realize the precise and quick pouring of drink from the source containers to various targeted containers with reliable performance, adaptability, manipulability, and explainability. [ABSTRACT FROM AUTHOR]