Search

Surface defect is a serious problem in real-world manufacturing system and it is important to use vision-based recognition to ensure the surface quality of products. Currently, due to the ability of automatic feature extraction, deep learning models, such as convolutional neural network (CNN), have been widely used in this area. However, these CNN-based models may not solve a problem well - inter-class similarities and intra-class variations (ISIV), which affect their ability of feature extraction and thus influence their recognition performance. To address this problem, this paper introduces a graph guidance mechanism into CNN to improve the ability of feature extraction, called Graph guided Convolutional Neural Network (GCNN). Firstly, GCNN defines a graph by computing the similarities between training samples. Secondly, the graph is introduced into VGG11, a popular CNN structure, to increase the inter-class distances and decrease the intra-class distances between defect samples. Meanwhile, a learnable coefficient is introduced into the training process to balance the effect of graph guidance automatically. The experimental results on four famous surface defect datasets demonstrate that the graph guidance helps CNN models have better ability of feature extraction and thus achieve better performance. Compared with state-of-the-art models, the proposed method can achieve the best performance. Furthermore, the final discussion shows that the learnable coefficient can help the proposed model to gain better performance, and that the proposed model increases a little computation cost compared to its original CNN model. Note to Practitioners—This paper is motivated by the problem in real-world manufacturing process – inter-class similarities and intra-class variations. Most of current CNN-based models may not solve this problem well, which limits their applications in real-world system. This paper proposes a graph guidance mechanism and introduces it to the training of CNN. The proposal can improve the ability of feature extraction, and thus have better performance than those without the graph guidance. By applying the proposal to four examples, the results show that the proposal is more feasible and effective than the state-of-the-art models in surface defect recognition. Furthermore, the mechanism assists the training of CNN and is removed in the process of recognition, so it will not increase time and occupied memory in recognition, and meanwhile, only limited computation cost is increased in training process. [ABSTRACT FROM AUTHOR]

The problem under consideration in this paper is how to avoid deadlocks and livelocks in the paradigm of Petri nets (PNs). Although deadlock and livelock avoidance has been extensively studied in existing literature, fewer results can be applied to practical automated manufacturing systems (AMSs) due to expensive computations and reduced permissiveness. In this paper, we propose an efficient and maximally permissive control scheme to avoid deadlocks and livelocks by using critical distance. From the perspective of the reachability graph of a PN, critical distance is equal to the maximum number of transitions among all transition sequences from any critical state to its corresponding deadlock or livelock state. First, we show how to calculate the critical distance of a PN in an efficient way by using a portion of reachable states. Then, local reachability graphs are established based on critical distance to avoid deadlocks and livelocks. The provided policy is maximally permissive since only the necessary minimum number of illegal transitions are forbidden at each state. Several representative examples are presented to illustrate our approach. Note to Practitioners—Deadlock and livelock avoidance is a crucial problem in automated manufacturing systems (AMSs). Various methods have been proposed to deal with deadlocks and livelocks by researchers and practitioners. However, there exist fewer works that focus on deep exploration regarding how long a deadlock or livelock will occur from a safe state. As a consequence, the existing works suffer from the disadvantages of being too aggressive or conservative, i.e., checking the whole state space or forbidding many acceptable states when avoiding deadlocks and livelocks. Through the analysis of a partial of reachable states, this paper derives the critical distance of Petri nets modeling AMSs, which is equal to the maximum number of transitions among all transition sequences from any critical marking to its corresponding deadlock or livelock marking. All deadlocks and livelocks, if exist, can be detected at any given marking within the critical distance. Our approach not only is maximally permissive but also explores only a partial of states, thereby mitigating state explosion problem. [ABSTRACT FROM AUTHOR]

To accurately pour drinks into various containers is an essential skill for service robots. However, drink pouring is a dynamic process and difficult to model. Traditional deep imitation learning techniques for implementing autonomous robotic pouring have an inherent black-box effect and require a large amount of demonstration data for model training. To address these issues, an Explainable Hierarchical Imitation Learning (EHIL) method is proposed in this paper such that a robot can learn high-level general knowledge and execute low-level actions across multiple drink pouring scenarios. Moreover, with the EHIL method, a logical graph can be constructed for task execution, through which the decision-making process for action generation can be made explainable to users and the causes of failure can be traced out. Based on the logical graph, the framework is manipulable to achieve different targets while the adaptability to unseen scenarios can be achieved in an explainable manner. A series of experiments have been conducted to verify the effectiveness of the proposed method. Results indicate that EHIL outperforms the traditional behavior cloning method in terms of success rate, adaptability, manipulability, and explainability. Note to Practitioners—Pouring liquids is a common activity in people’s daily lives and all wet-lab industries. Drink pouring dynamic control is difficult to model, while the accurate perception of flow is challenging. To enable the robot to learn under unknown dynamics via observing the human demonstration, deep imitation learning can be used. To address the limitations of traditional deep neural networks, an Explainable Hierarchical Imitation Learning (EHIL) method is proposed in this paper. The proposed method enables the robot to learn a sequence of reasonable pouring phases for performing the task rather than simply execute the task via traditional behavior cloning. In this way, explainability and safety can be ensured. Manipulability can be achieved by reconstructing the logical graph. The target of this research is to obtain pouring dynamics via the learning method and realize the precise and quick pouring of drink from the source containers to various targeted containers with reliable performance, adaptability, manipulability, and explainability. [ABSTRACT FROM AUTHOR]

To achieve the better navigation performance of a mobile robot in the unknown environments, a novel human–robot hybrid system incorporating a motor-imagery (MI)-based brain teleoperation control is presented in this paper, where a deep-learning-based active perception is developed in the simultaneous localization and mapping (SLAM) framework. Using the deep-learning-based object recognition in the red–green–blue-depth (RGB-D) data acquisition process, the designed SLAM approach can select the valid feature points effectively, and the speed of displacement tracking can be improved by combining the oriented FAST and rotated BRIEF (ORB) SLAM algorithm with the optical flow method. The global trajectory map can also be mended using graph-based nonlinear error optimization. In addition, to build the connection between human intentions and the robot control commands flexibly in the developed mobile robot, a common spatial pattern (CSP)-based support vector machine (SVM) classification algorithm is proposed so that the control commands can be obtained directly from the human electroencephalograph (EEG) signals, which are preanalyzed and classified using the phenomena of event-related synchronization/desynchronization (ERS/ERD). Experiments involving several operators have verified the effectiveness of the proposed framework in the actual unstructured environments. Note to Practitioners—This paper is motivated by the exploration of the mobile robots remotely controlled by the disables in an unstructured environment that includes some unknown moving objects in the background. In the conventional approaches, the image feedback and the brain–computer interface-based EEG signals invoked by the visual stimulus are used to guide the motion of the robots. In this paper, the MI of left or right hand is adopted as an input of the brain teleoperation system, and the EEG signals are classified into two categories representing two robot control commands through the designed algorithm. Besides, a deep-learning-based SLAM is designed to analyze the image and depth of the environmental information provided by the robot RGB-D sensor, and the results can be transferred into a 3-D map locating the robot and assisting the operator to understand the operating environment. It has been verified that the deep-learning-based SLAM presented is more efficient and robust than traditional SLAM. The feasibility of the system is demonstrated by a set of experiments in the corridor environment. [ABSTRACT FROM AUTHOR]

This paper presents a novel computational intelligence vision sensing approach to estimate nutrient content in wheat leaves by analyzing color features of the leaves images captured on field with various lighting conditions. We propose the development of deep sparse extreme learning machines (DSELM) fusion and genetic algorithm (GA) to normalize plant images as well as to reduce color variability due to a variation of sunlight intensities. We also apply the DSELM in image segmentation to differentiate wheat leaves from a complex background. In this paper, four moments of color distribution of the leaves images (mean, variance, skewness, and kurtosis) are extracted and utilized as predictors in the nutrient estimation. We combine a number of DSELMs with committee machine and optimize them using the GA to estimate nitrogen content in wheat leaves. The results have shown the superiority of the proposed method in the term of quality and processing speed in all steps, i.e., color normalization, image segmentation, and nutrient prediction, as compared with other existing methods. Note to Practitioners—In order to support precision farming technology and agricultural automation, it is very essential to estimate the nutrient content in plants to prevent over fertilizing that will harm environment. Furthermore, the existing method to determine nutrient content in leaves is destructive as well as time consuming and requires special expertise to operate expensive devices. A number of methods have been developed to estimate nutrient content in leaves nondestructively based on the color features of the leaves. Most of these methods, however, are conducted in a controlled environment with artificial lighting. Such methods are not practical and need various equipment. We propose a low-cost, simple, and accurate technique to estimate nitrogen content in wheat leaves by analyzing RGB color of the leaves images. In this paper, we found that the developed DSELMs fusion has enabled better performance in normalizing images and is faster than other neural network types, i.e., backpropagation-based multilayer perceptron and original extreme learning machine. In image segmentation step, the established DSELM shows good performance to recognize and distinguish wheat leaves from other undesired background images. Furthermore, the developed weighted DSELMs have demonstrated enhanced ability in estimating nutrient content. [ABSTRACT FROM AUTHOR]

In manufacturing industries, it is of fundamental importance to detect anomalies in production in order to meet the required quality goals and to limit the number of defective products that are accidentally delivered to the customers. Nevertheless, monitoring systems currently employed in production are typically very simple and rely on a set of univariate control charts that fail to capture the multivariate and complex nature of real-world industrial systems. In such context, Machine Learning (ML)-based approaches for Anomaly Detection (AD) have proven to be extremely effective in increasing anomalies detectability and, in general, in enhancing monitoring procedures. However, industrial data are typically very complex and not suitable to be fed directly to classical ML-based AD tools making feature extraction procedures a necessary step that unfortunately may lead to information loss and low scalability. Deep Learning, has proven very effective at learning useful representations of complex data in an automatic way. In this paper, we propose an AD pipeline that makes use of convolutional autoencoders to extract useful features from two-dimensional, non-image, data. We test our approach on real world Optical Emission Spectroscopy data that are typical of semiconductor manufacturing and we achieve improved performance over classical monitoring methods. Note to Practitioners—Advanced monitoring is one of the most important task in the context of Industry 4.0. Some of the main issues in developing Machine Learning-based solutions in industrial environment are: (i) the lack of reliable tagged data; (ii) the complexity of data structures present in real-world scenarios. In this paper we investigate unsupervised anomaly detection for 2-dimensional data in manufacturing environment: we provide an approach that exploit Deep Learning-based architecture for handling the data at hand. We show the effectiveness of the proposed approach in a real world case study related to optical emission spectroscopy data in semiconductor manufacturing process providing satisfactory classification accuracy. [ABSTRACT FROM AUTHOR]

Predictive Maintenance technologies are particularly appealing for Industrial Equipment producers, as they pave the way to the selling of high added-value services and customized maintenance plans. However, standard Predictive Maintenance approaches assume the availability of sensor measurements, and the costs associated with adding sensors or remotely accessing sensor readings may discourage the development of such technologies. In this context, Alarm Forecasting can be very useful as it represents a low-cost alternative or helpful support to sensor-based Predictive Maintenance. In this work, we propose a new formulation for the Alarm Forecasting problem, framed as a multi-label classification task. We present a novel deep learning-based approach called FORMULA (alarm FORecasting in MUlti-LAbel setting). FORMULA leverages Transformer, a popular Neural Network architecture in the field of Natural Language Processing. To cope with alarm imbalance, we draw inspiration from Segmentation and Object Detection. Thus, FORMULA is trained by minimizing the Weighted Focal Loss, which turns out to be very effective in predicting rare alarms. These alarms, even if they are difficult to predict by nature, often are business-critical. We assess the proposed approach on a representative real-world problem from the packaging industry. In particular, we show that it outperforms not only classic multilabel techniques but also models based on recurrent neural networks. As regards the latter, the proposed approach also exhibits a lower computational burden, both in terms of training time and model size. To foster research in the field and reproducibility, we also publicly share the alarm logs dataset and the code used to perform the experiments. Note to Practitioners—This paper was motivated by the problem of monitoring equipment in the scenario of dairy products packaging, under the mild assumption that logs of the alarm generated by the packaging machines are available. This paper proposes an alarm forecasting algorithm. Its goal is to predict if any alarm will occur in the future, based only on past alarm logs. The limits of the considered future window can be defined arbitrarily, so there is enough time to perform corrective actions. Thus, the proposed approach aims to prevent unexpected downtime that would not only hinder productivity but also imply significant material waste. The proposed approach leverages methodologies from Natural Language Processing and Object Detection to deal with rare alarms that are often very informative in the industrial scenario. Besides, both the code and the real-world industrial datasets used to evaluate the methodology are available publicly. Currently, the proposed approach only uses alarm logs. Especially in the context of Industry 4.0, where many sensory data may be available, this is a limitation. Thus, the described approach might be extended by integrating alarm logs with sensory data. This integration is expected to improve the estimation of equipment health state. The results described in this paper may find application not only in the manufacturing sector but also in different areas such as Cyber Security, where log files keep activity records of each process performed. [ABSTRACT FROM AUTHOR]

The aspect ratio of a target changes frequently during an unmanned aerial vehicle (UAV) tracking task, which makes the aerial tracking very challenging. Traditional trackers struggle from such a problem as they mainly focus on the scale variation issue by maintaining a certain aspect ratio. In this paper, we propose a coarse-to-fine deep scheme to address the aspect ratio variation in UAV tracking. The coarse-tracker first produces an initial estimate for the target object, then a sequence of actions are learned to fine-tune the four boundaries of the bounding box. The coarse-tracker and the fine-tracker are designed to have different action spaces and operating target. The former dominates the entire bounding box and the latter focuses on the refinement of each boundary. They are trained jointly by sharing the perception network with an end-to-end reinforcement learning architecture. Experimental results on benchmark aerial data set prove that the proposed approach outperforms existing trackers and produces significant accuracy gains in dealing with the aspect ratio variation in UAV tracking. Note to Practitioners—During the past years, unmanned aerial vehicle (UAV) have gained much attention for both industrial and consumer uses. It is in urgent demand to endow the UAV with intelligent vision-based techniques, and the automatic target following via visual tracking methods as one of the most fundamental intelligent features could promote various applications of UAVs, such as surveillance, augmented reality, and behavior modeling. Nonetheless, the primary issue of a UAV-based tracking method is the platform itself: it is not stable, it tends to have sudden movements, it generates nonhomogeneous data (scale, angle, rotation, depth, and so on), all of them tend to change the aspect ratio of the target frequently and further increase the difficulty of object tracking. This paper aims to address the aspect ratio change (ARC) problem in UAV tracking. We present a coarse-to-fine strategy for UAV tracking. Specifically, the coarse bounding box is obtained to locate the target firstly. Then, a refinement scheme is performed on each boundary to further improve the position estimate. The tracker is proved to be effective to increase the resistance to the ARC. Such a method can be implemented on UAV to improve the target-following performance. [ABSTRACT FROM AUTHOR]

Under the consideration of massive energy consumption of machine tools, many approaches have been proposed, and state control method of machine tools has proved its effectiveness. In order to satisfy the demand of real-time production control, a deep learning methodology for energy-efficient control of CNC machine tools is proposed in RFID-enabled ubiquitous environment. First, the energy-efficient control strategies for multiple machine tools are proposed to reduce the carbon emission of the machining process. Then, through evaluating the process progress in the RFID-enabled environment, a deep learning methodology for energy-efficient strategies selection of CNC machine tools using deep belief networks (DBNs) is established to realize the real-time and accurate control of machine tools. Finally, comparisons between the proposed approach and some state-of-the-art ones are given, and the experiment results indicate that the proposed method is effective and efficient for the energy-efficient control problem of machine tools. The proposed method can realize the real-time control of CNC machine tools based on the interaction information in Industrial 4.0. Furthermore, the machine tools will be converted to smart machines, which can complete self-perception and self-adjustment automatically. Note to Practitioners—It is significant but challenging work to realize the control of manufacturing processes based on real-time production data. Thus, this paper integrates RFID data of jobs with energy-efficient control of CNC machine tools, and proposes a deep learning methodology of processing the real-time production data in an RFID-enabled ubiquitous environment. Considering the relationship between different jobs, five energy-efficient control strategies for multiple machine tools are put forward to reduce the carbon emission of the machining processes. Then, an RFID-driven process progress evaluation is carried out to quantify the real-time progress, and two RFID data preprocessing algorithms are developed to cleanse and extract the original data. DBNs are adopted to realize the energy-efficient strategies selection of CNC machine tools. The experiments from an actual printing machine manufacturing enterprise indicate that the proposed method is effective and efficient for the energy-efficient control problem of machine tools. The implementation requires RFID-enabled manufacturing environment deployment, RFID data capturing and prepressing, and deep learning model establishment based on historical production data. Beyond the energy conservation of machine tools, the proposed method can also be applied to other industrial problems, e.g., self-perception and self-adjustment of smart machine tools, production rescheduling decisions, and logistics routing optimization. [ABSTRACT FROM AUTHOR]

Video and image sources are frequently applied in the area of defect inspection in industrial community. For the recognition and classification of sewer defects, a significant number of videos and images of sewers are collected. These data are then checked by human and some traditional methods to recognize and classify the sewer defects, which is inefficient and error-prone. Previously developed features like SIFT are unable to comprehensively represent such defects. Therefore, feature representation is especially important for defect autoclassification. In this paper, we study the automatic extraction of feature representation for sewer defects via deep learning. Moreover, a complete automatic system for classifying sewer defects is proposed built on a two-level hierarchical deep convolutional neural network, which shows high performance with respect to classification accuracy. The proposed network is trained on a novel data set with over 40 000 sewer images. The system has been successfully applied in the practical production, confirming its robustness and feasibility to real-world applications. The source code and trained model are available at the project website. Note to Practitioners—Automatic defect inspection has become a fundamental research topic in engineering application field. Specifically, sewer defect detection is an important measure for maintenance, renewal, and rehabilitation activities of sewer infrastructure. In the current operation procedure, all the captured videos need to be inspected by experts frame by frame to recognize defects, yielding a significant low inspection rate with a significant amount of time. Previous work has attempted to employ traditional image processing methods for automated sewer defect classification. However, these methods get poor generalization capabilities since they use pre-engineered features. In most cases, sewerage inspection companies have to hire numerous professional inspectors to do this job, thereby consuming a lot of human and material resources. To address this problem, the authors propose an automatic detection and classification method for sewer defects based on hierarchical deep learning. Demonstrated by various experiments, the designed framework achieves a high defect classification accuracy, which can be easily integrated into an automatic sewer defect inspection system. 1 https://github.com/NUAAXQ/SewerDefectDetection [ABSTRACT FROM AUTHOR]

In this paper, we propose a discriminative representation for patterned fabric defect detection when only limited negative samples are available. Fabric patches are efficiently classified into defectless and defective categories by Fisher criterion-based stacked denoising autoencoders (FCSDA). First, fabric images are divided into patches of the same size, and both defective and defectless samples are utilized to train FCSDA. Second, test patches are classified through FCSDA into defective and defectless categories. Finally, the residual between the reconstructed image and defective patch is calculated, and the defect is located by thresholding. Experimental results demonstrate the effectiveness of the proposed scheme in the defect detection for periodic patterned fabric and more complex jacquard warp-knitted fabric. [ABSTRACT FROM AUTHOR]

Despite rapid advances in machine learning based fault diagnosis, their identical distribution assumption of the training (source domain) and testing data (target domain) is generally challenged in industrial applications due to the variation of working conditions. In this article, a fault-prototypical adapted network (FPAN) is proposed, which enables cross-domain industrial intelligent fault diagnosis aided by deep transfer learning. First, a similarity learning-based discrimination module is designed to learn fault prototypes (FPs) that are representative for each fault and discriminative across different faults. Then, a fault prototypical-adaptation module is developed, which adapts the multiple FPs to the target dataset and enables more precise category-wise domain invariance. The two modules are trained simultaneously to extract transferrable and discriminative FPs, by which the cross-domain intelligent diagnosis can be readily achieved. Experimental results on two industrial cases illustrate that the proposed approach learns transferable feature representations that better reduce domain discrepancy, and provides improved diagnosis performance on target data. Note to Practitioners—As the distribution of source and target domain data may differ due to the varying working conditions, cross-domain intelligent fault diagnosis plays an increasingly important role in practical industrial cases. This work presents the FPAN model, in which the fault discrimination and the category-wise adaptation are naturally connected and unified by exploring representative virtual FP for each category, irrespective of which domain that the category comes from. Connected by the adaptive FPs, the similarity learning-based discrimination and the fault-prototypical adaptation modules can benefit from each other to boost the target learning. Finally, the cross-domain fault diagnosis can be performed based on the similarities with different FP representations. The proposed method is readily applicable to various intelligent fault diagnosis problems, including the mechanical cases and the industrial processes. [ABSTRACT FROM AUTHOR]

Some response surface functions in complex engineering systems are usually highly nonlinear, unformed, and expensive to evaluate. To tackle this challenge, Bayesian optimization (BO), which conducts sequential design via a posterior distribution over the objective function, is a critical method used to find the global optimum of black-box functions. Kernel functions play an important role in shaping the posterior distribution of the estimated function. The widely used kernel function, e.g., radial basis function (RBF), is very vulnerable and susceptible to outliers; the existence of outliers is causing its Gaussian process (GP) surrogate model to be sporadic. In this article, we propose a robust kernel function, asymmetric elastic net radial basis function (AEN-RBF). Its validity as a kernel function and computational complexity are evaluated. When compared with the baseline RBF kernel, we prove theoretically that AEN-RBF can realize smaller mean squared prediction error under mild conditions. The proposed AEN-RBF kernel function can also realize faster convergence to the global optimum. We also show that the AEN-RBF kernel function is less sensitive to outliers, and hence improves the robustness of the corresponding BO with GPs. Through extensive evaluations carried out on synthetic and real-world optimization problems, we show that AEN-RBF outperforms the existing benchmark kernel functions. Note to Practitioners—Some industrial systems cannot be accurately represented by physical models. In this situation, data-driven black-box optimization is necessary for advancing the system automation and intelligence. BO is one of the widely used strategies for learning the global optimum of black-box functions. BO has been applied to robotics, anomaly detection, automatic learning algorithm configuration, reinforcement learning, and deep learning. This article proposes one new kernel function, named after AEN-RBF. The new kernel function will make BO with GPs more robust to outliers and lower the data quality barrier of model training. This article was motivated by the hyperparameter tuning problem of deep learning models for image defect detection in advanced manufacturing, but the method can be easily extended to other applications where kernel functions are needed. Our proposed method is verified by synthetic and real-world optimization problems. [ABSTRACT FROM AUTHOR]

Accurate and real-time prediction of network traffic can not only help system operators allocate resources rationally according to their actual business needs but also help them assess the performance of a network and analyze its health status. In recent years, neural networks have been proved suitable to predict time series data, represented by the model of a long short-term memory (LSTM) neural network and a temporal convolutional network (TCN). This article proposes a novel hybrid prediction method named SG and TCN-based LSTM (ST-LSTM) for such network traffic prediction, which synergistically combines the power of the Savitzky–Golay (SG) filter, the TCN, as well as the LSTM. ST-LSTM employs a three-phase end-to-end methodology serving time series prediction. It first eliminates noise in raw data using the SG filter, then extracts short-term features from sequences applying the TCN, and then captures the long-term dependence in the data exploiting the LSTM. Experimental results over real-world datasets demonstrate that the proposed ST-LSTM outperforms state-of-the-art algorithms in terms of prediction accuracy. Note to Practitioners—This work considers real-time and high-accuracy prediction of network traffic. It is highly important to well predict network traffic by capturing long-term dependence and effectively extracting high- and low-frequency information from time series data. Yet, it is a big challenge to achieve it because there are unstable characteristics and strong nonlinear features in the network traffic due to continuous expansion of network scale and fast emergence of new services. Current prediction methods usually have oversimplified theoretical assumptions, need significant time and memory, or suffer problems of gradient disappearance or early convergence. Thus, they fail to effectively capture the nonlinear characteristics of large-scale network sequences. This work proposes a hybrid prediction method named SG and TCN-based LSTM (ST-LSTM), which integrates the merits of the Savitzky–Golay filter, the temporal convolutional network (TCN), and the long short-term memory (LSTM), serving as smoothing time series, capturing short-term local features, and capturing long-term dependence, respectively. Experimental results based on the real-life dataset demonstrate that it achieves better prediction accuracy than its state-of-the-art peers, including the TCN and the LSTM. It can be readily implemented and deployed in many real-life industrial areas including smart city, edge computing, cloud computing, and data centers. [ABSTRACT FROM AUTHOR]

Deep belief network (DBN) is an effective learning model based on deep learning. It can hierarchically transform the input data via stacked feature detectors. As a predictive model, DBN has shown a promising prospect in model predictive control (MPC). However, its successful application relies seriously on the suitable structure size (the numbers of hidden layers and neurons), which is challenging to determine. In this work, we present a theoretical bound on its minimum structure size in order to accurately approximate a desired control law of MPC, called DBN-MPC. First, according to the Markov assumption, a controlled system is equivalent to a quadratic program, which only depends on the current system state. Second, a universal theorem is proposed to give a bound on the minimum structure size of DBN from the perspective of piecewise affine function analysis. Third, a partial least square regression is used to fine-tune DBN to overcome the problems of local-minimum and time-consuming training process. Finally, we demonstrate the effectiveness of the proposed method through two classical experiments: 1) tracking control of a benchmark dynamical system and 2) temperature control of a practical second-order continuous stirred tank reactor (CSTR) system. The experimental results generally give an answer to the question that how deep is deep enough for DBN to approximate an MPC law. Note to Practitioners—The proposed deep belief network (DBN)-model predictive control (MPC) scheme presents an answer to the question that how deep is deep enough for DBN, from the perspective of approximating an MPC law. Given an industrial system, the proposed DBN-MPC scheme can give the best tracking control performance with minimal computational complexity through an effective self-growing DBN and optimal approximation theorem of MPC law. In practical applications, it is better for practitioners to obtain accurate historical input and output data. During the operation process, DBN-MPC will be more practical if more accurate initial structure size of DBN is provided in advance because the initial structure size of DBN is the starting point of the dynamic optimization. [ABSTRACT FROM AUTHOR]

In the real-time status monitoring of pipeline network, incomplete pressure data are unavoidable due to some device or communication errors. To solve this problem, a hierarchical data recovery method based on generative adversarial networks (GANs) is proposed in this article. First, a hierarchical data recovery framework is proposed to handle different numbers of incomplete data due to the structure of the semicentral pipeline network. Second, a joint attention module is presented to capture both interior nature and correlation relationships of multivariate pressure series and further guarantee the consistency of pressure data. Third, the macromicrodual discriminators are proposed to evaluate the recovery result through the combination of the local and global variation in temporal and spatial dependencies. Based on the novel structures, the proposed model is able to recover incomplete data with abnormal fluctuation values, unreasonable fixed values, or missing values. Finally, under a series of data recovery experiments, the efficiency of the proposed method is evaluated. Experimental results demonstrate that the proposed method is a practical way to ensure data recovery performance in the pipeline network. Note to Practitioners—Status monitoring based on pressure data is of great importance for safe and efficient operation in a pipeline network. However, due to unexpected situations, the appearance of incomplete pressure data affects the subsequent data processing and status analysis, resulting in an incorrect decision. In this article, a deep learning-based method is proposed to recover the incomplete data. With the help of the spatiotemporal dependencies of multivariate pressure series, the proposed method can recover different numbers of incomplete data through the no-missing part of pressure data. The experiment results show that the proposed method is better than the similar data recovery methods through three different evaluation metrics. In the future, we will address the data recovery problem without the complete data pairs in the training process. [ABSTRACT FROM AUTHOR]

For extrusion-based additive manufacturing, the variation in material deposition can significantly affect printed material distribution, causing infill nonuniformity and defects. These variations are induced by kinematic variations of the printer extruder. Such infill nonuniformity is more significant in an application of collaborative printing systems by which multiple printers’ extrudes co-create the same structure since more accelerate–decelerate kinematic cycles are involved. There is a lack of a quantitative understanding of the impact of printing kinematics on such variations to guide the printing process control. This article deals with the challenge by establishing a mathematical model that quantifies the printing width variations along the printing paths induced by printing speed and acceleration. The model provides vital information for predicting infill pattern nonuniformity and potentially enables using G-code adjustment to compensate for the infill errors in future research. In addition, since the model captures the mechanism of kinematics-induced variations, it provides a way of between-printer knowledge transfer on estimating printing errors. This article further proposes an informative-prior-based transfer learning algorithm to improve the quality prediction model for a printer with limited historical data by leveraging the shared data from interconnected 3-D printers. A case study based on experiments validated the effectiveness of the proposed methodology. Note to Practitioners—This article quantitatively studies the impact of extruder kinematics on geometric variations and printing quality in extrusion-based 3-D printing processes. The model can help predict the geometric printing quality and related defects, such as overfill or underfill problems given kinematics setup by G-code. This study can expedite the learning process of printing variations induced by kinematics for new printers to set up monitoring and G-code adjustment for process control in the early stage of production when the data are limited. In the long run, such between-printer transfer learning has the potential to enable the transfer learning for interconnected collaborative 3-D printing systems with improved printing efficiency and quality. [ABSTRACT FROM AUTHOR]

Smart structural health monitoring (SHM) for large-scale infrastructure is an intriguing subject for engineering communities thanks to its significant advantages such as timely damage detection, optimal maintenance strategy, and reduced resource requirement. Yet, it is a challenging topic as it requires handling a large amount of collected sensors data continuously, which is inevitably contaminated by random noises. Therefore, this study developed a practical end-to-end framework that makes use of physical features embedded in raw data and an elaborated hybrid deep learning model, namely 1-DCNN-LSTM, featuring two algorithms—convolutional neural network (CNN) and long-short term memory (LSTM). In order to extract relevant features from sensory data, the method combines various signal processing techniques such as the autoregressive model, discrete wavelet transform, and empirical mode decomposition. The hybrid deep learning 1-DCNN-LSTM is designed based on the CNN’s capacity of capturing local information and the LSTM network’s prominent ability to learn long-term dependencies. Through three case studies involving both experimental and synthetic data sets, it is demonstrated that the proposed approach achieves highly accurate damage detection, as accurate as the powerful 2-D CNN, but with a lower time and memory complexity, making it suitable for real-time SHM. Note to Practitioners—This article aims to develop a practical data-driven method for automatically monitoring the operational state of structures. In order to achieve consistently and highly accurate results in performing different tasks for diverse structures, we combine underlying features in both time and frequency domains extracted from measured signal vibration data. Three popular data featuring methods are combined to achieve the diversity gain which would not be possible with each individual method. As the vibration is usually measured by long time-series signals, the most efficient deep learning architecture for time-series signal, namely long-short term memory (LSTM), is considered for this work. Besides, each structure has its own dynamic properties, i.e., eigenfrequencies, around which the most relevant information is in the frequency domain, thus convolutional neural network specifically designed for capturing local information is used in combination with LSTM, forming a hybrid deep learning architecture. The applicability and effectiveness of the proposed approach are supported by three case studies with different types of structures, showing highly accurate damage detection with reduced resource requirements. These advantages can be valuable for developing a model for live monitoring of structural health in the future life-line infrastructures. [ABSTRACT FROM AUTHOR]

Semantic segmentation of urban scenes is an essential component in various applications of autonomous driving. It makes great progress with the rise of deep learning technologies. Most of the current semantic segmentation networks use single-modal sensory data, which are usually the RGB images produced by visible cameras. However, the segmentation performance of these networks is prone to be degraded when lighting conditions are not satisfied, such as dim light or darkness. We find that thermal images produced by thermal imaging cameras are robust to challenging lighting conditions. Therefore, in this article, we propose a novel RGB and thermal data fusion network named FuseSeg to achieve superior performance of semantic segmentation in urban scenes. The experimental results demonstrate that our network outperforms the state-of-the-art networks. Note to Practitioners—This article investigates the problem of semantic segmentation of urban scenes when lighting conditions are not satisfied. We provide a solution to this problem via information fusion with RGB and thermal data. We build an end-to-end deep neural network, which takes as input a pair of RGB and thermal images and outputs pixel-wise semantic labels. Our network could be used for urban scene understanding, which serves as a fundamental component of many autonomous driving tasks, such as environment modeling, obstacle avoidance, motion prediction, and planning. Moreover, the simple design of our network allows it to be easily implemented using various deep learning frameworks, which facilitates the applications on different hardware or software platforms. [ABSTRACT FROM AUTHOR]

Compared with the rapidly growing artificial intelligence (AI) research in other branches of healthcare, the pace of developing AI capacities in dental care is relatively slow. Dental care automation, especially the automated capability for dental cone beam computed tomography (CBCT) segmentation and lesion detection, is highly needed. CBCT is an important imaging modality that is experiencing ever-growing utilization in various dental specialties. However, little research has been done for segmenting different structures, restorative materials, and lesions using deep learning. This is due to multifold challenges such as content-rich oral cavity and significant within-label variation on each CBCT image as well as the inherent difficulty of obtaining many high-quality labeled images for training. On the other hand, oral-anatomical knowledge exists in dentistry, which shall be leveraged and integrated into the deep learning design. In this article, we propose a novel anatomically constrained Dense U-Net for integrating oral-anatomical knowledge with data-driven Dense U-Net. The proposed algorithm is formulated as a regularized or constrained optimization and solved using mean-field variational approximation to achieve computational efficiency. Mathematical encoding for transforming descriptive knowledge into a quantitative form is also proposed. Our experiment demonstrates that the proposed algorithm outperforms the standard Dense U-Net in both lesion detection accuracy and dice coefficient (DICE) indices in multilabel segmentation. Benefited from the integration with anatomical domain knowledge, our algorithm performs well with data from a small number of patients included in the training. Note to Practitioners—This article proposes a novel deep learning algorithm to enable the automated capability for cone beam computed tomography (CBCT) segmentation and lesion detection. Despite the growing adoption of CBCT in various dental specialties, such capability is currently lacking. The proposed work will provide tools to help reduce subjectivity and human errors, as well as streamline and expedite the clinical workflow. This will greatly facilitate dental care automation. Furthermore, due to the capacity of integrating oral-anatomical knowledge into the deep learning design, the proposed algorithm does not require many high-quality labeled images to train. The algorithm can provide good accuracy under limited training samples. This ability is highly desirable for practitioners by saving labor-intensive, costly labeling efforts, and enjoying the benefits provided by AI. [ABSTRACT FROM AUTHOR]

Smart customization service is an important element for smart manufacturing. The success of smart customization requires that designers, manufacturers, and customers with differences in context, semantics, and other cognitive aspects be engaged in a collaborative process. With product configurators reported to have positive impacts on product quality to meet customers’ needs, this article attempts to explore an approach for smart customization service based on configurators. To better address the semantic gap between customers and designers/manufacturers, a new configuration mechanism is proposed that takes into consideration customer needs using natural language as the input and maps them to product specifications in the design stage. We collected a massive amount of review text from e-commerce websites and used ELMo, a contextualized word representation based on a deep bidirectional language model, to encode the text. A multitask learning-based neural network was adopted to build the mapping from layman customer needs to product specifications. Our experiments show that this approach can achieve a promising performance for the configuration task and, thereby, facilitate smart customization services. Note to Practitioners—Smart customization has been adopted by various industries to tailor companies’ business streams and customize solutions for customers. It is a complicated service process involving cross-functional teams for identifying customer needs and establishing product design and manufacturing specifications. However, communication in the collaborative process may be challenging. Customers may express their needs using layman’s terms. The expressed needs can even be ambiguous and imprecise. Miscommunication hinders the efficiency of smart customization services. This article uses natural language processing and machine learning techniques to map product review text, which is crawled from e-commerce websites to the corresponding product specifications. Given a new customer’s needs in free-form text, the mapping can automatically identify the satisfactory product configurations. This has the potential to improve the efficiency of product customization and shield customers and companies from the back-and-forth communication procedure in the customization service. [ABSTRACT FROM AUTHOR]