Search

In this paper we introduce the current thrust of development in machine learning and artificial intelligence, fueled by advances in statistical learning theory over the last 20 years and commercial successes by leading big data companies. Then we discuss the characteristics of process manufacturing systems and briefly review the data analytics research and development in the last three decades. We give three attributes for process data analytics to make machine learning techniques applicable in the process industries. Next we provide a perspective on the currently active topics in machine learning that could be opportunities for process data analytics research and development. Finally we address the importance of a data analytics culture. Issues discussed range from technology development to workforce education and from government initiatives to curriculum enhancement. [ABSTRACT FROM AUTHOR]

Highlights • A critical overview of batch data analysis (BDA) methods is provided. • BDA methods are mapped into the complexity scale. • Both modeling complexity (method's degree of freedom) and implementation complexity (expertise/training required) are considered. • The complexity scale can be applied to select a suitable method for a given task. • The complexity scale can also guide company-wide training programs. Abstract More than two decades have passed since the first holistic data-driven approaches for batch data analysis (BDA) were published. The emphasis was on multivariate statistical process monitoring and quality prediction. In the subsequent years, more methods were proposed, with varying degrees of success and acceptance by practitioners. A detailed and comprehensive analysis of these contributions reveals different complexity levels, both in terms of the degrees of freedom used for setting up the techniques (modeling complexity) and the expertise/training required by practitioners to autonomously apply them in concrete real world applications (implementation complexity). Both dimensions decisively contribute to the impact of a given proposal in industry. In this paper, we present a structured overview of BDA methodologies and analyze them from both perspectives. As a corollary, we map the BDA methods into the complexity scale, and elaborate how it can be used for selecting a suitable method for a given task. [ABSTRACT FROM AUTHOR]

Feature extraction is crucial for fault diagnosis and the use of complementary features allows for improved diagnostic performance. Most of the existing fault diagnosis methods only utilize data-driven and causal connectivity-based features of faults, whereas the important complementary feature of the propagation paths of faults is not incorporated. The propagation path-based feature is important to represent the intrinsic properties of faults and plays a significant role in fault diagnosis, particularly for the diagnosis of multiple and unknown faults. In this article, a three-step framework based on the modified distance (DI) and modified causal dependency (CD) is proposed to integrate the data-driven and causal connectivity-based features with the propagation path-based feature for diagnosing known, unknown, and multiple faults. The effectiveness of the proposed approach is demonstrated on the Tennessee Eastman process. [ABSTRACT FROM AUTHOR]

Abstract: Dow Chemical is committed to product quality consistency. To achieve this goal, a project on implementing multivariate quality control was launched. Robust principal component analysis (PCA) was applied to a historical data set of product quality to remove outliers. The remaining data, representing normal process variation, were used to build a PCA model. One advantage of multivariate analysis is that PCA can detect a change in variable correlation, which is undetectable using univariate control charts. The T 2 statistic is used for fault detection and the contribution chart is used for fault identification. The model has been implemented on-line and proven to be effective in detecting and identifying abnormal product lots and analytical issues. Practical issues such as long-term model performance, model maintenance, and model transfer are discussed in this paper. [Copyright &y& Elsevier]

Abstract: The data collected from a batch process over time from multiple sensors can be arranged in a matrix of J-variables× K-time points. Data collected on multiple batches can be arranged in a cube of I-batches× J-variables× K-time points. The analysis of a cube of data can be performed by unfolding in two different ways, batch unfolding giving an I × JK data matrix or observation unfolding resulting in an IK × J data matrix, followed by PCA. The data can also be analyzed directly using three-way methods such as PARAFAC or Tucker3. In the literature there is no clear agreement as to the most effective approach for the analysis of batch data. This paper makes detailed comparisons between the two unfolding approaches and the Tucker3 method. Batch data from a fermentation process at The Dow Chemical Company San Diego facility is used for this study. The three methods were found to be complementary to each other and a well-trained chemometrician/practitioner will find all three methods to be useful for batch data analysis. The batch unfolding MPCA is more sensitive to the overall batch variation while the observation unfolding MPLS is more sensitive to the localized batch variation. The Tucker3 method is in good balance in terms of detecting both variations. [Copyright &y& Elsevier]

The proficiencies of Fisher discriminant analysis (FDA), support vector machines (SVM), and proximal support vector machines (PSVM) for fault diagnosis (i.e. classification of multiple fault classes) are investigated. The Tennessee Eastman process (TEP) simulator was used to generate overlapping datasets to evaluate the classification performance. When all variables were used, the datasets were masked with irrelevant information, which resulted in poor classification. With key variables selected by genetic algorithms and the contribution charts, SVM and PSVM outperformed FDA and demonstrated the advantage of using nonlinear technique when data are overlapped. The overall misclassification for the testing data using FDA dropped from 38 to 18%; while those using SVM and PSVM dropped from 44–45 to 6%. The effectiveness of the proposed approach is increased in PSVM by saving significant computation time and memory requirement, while obtaining comparable classification results. For auto-correlated data, the incorporation of time lags into SVM and PSVM improved classification results. The added dimensions decreased the degree to which the data overlap and the overall misclassification for the testing set using SVM and PSVM decreased further to 3%. [Copyright &y& Elsevier]

Many trouble-shooting problems in process industries are related to key variable identification for classifications. The contribution charts, based on principal component analysis (PCA), can be applied for this purpose. Genetic algorithms (GAs) have been proposed recently for many applications including variable selection for multivariate calibration, molecular modeling, regression analysis, model identification, curve fitting, and classification. In this paper, GAs are incorporated with Fisher discriminant analysis (FDA) for key variable identification. GAs are used as an optimization tool to determine variables that maximize the FDA classification success rate for two given data sets. GA/FDA is a proposed solution for the variable selection problem in discriminant analysis. The Tennessee Eastman process (TEP) simulator was used to generate the data sets to evaluate the correctness of the key variable selection using GA/FDA, and the T2 and Q statistic contribution charts. GA/FDA correctly identifies the key variables for the TEP case studies that were tested. For one case study where the correlation changes in two data sets, the contribution charts incorrectly suggest that the operating conditions are similar. On the other hand, GA/FDA not only determines that the operating conditions are different, but also identifies the key variables for the change. For another case study where many key variables are responsible for the changes in the two data sets, the contribution charts only identifies a fraction of the key variables, while GA/FDA correctly identifies all of the key variables. GA/FDA is a promising technique for key variable identification, as is evidenced in successful applications at The Dow Chemical Company. [Copyright &y& Elsevier]

To implement on-line process monitoring techniques such as principal component analysis (PCA) or partial least squares (PLS), it is necessary to extract data associated with the normal operating conditions from the plant historical database for calibrating the models. One way to do this is to use robust outlier detection algorithms such as resampling by half-means (RHM), smallest half volume (SHV), or ellipsoidal multivariate trimming (MVT) in the off-line model building phase. While RHM and SHV are conceptually clear and statistically sound, the computational requirements are heavy. Closest distance to center (CDC) is proposed in this paper as an alternative for outlier detection. The use of Mahalanobis distance in the initial step of MVT for detecting outliers is known to be ineffective. To improve MVT, CDC is incorporated with MVT. The performance was evaluated relative to the goal of finding the best half of a data set. Data sets were derived from the Tennessee Eastman process (TEP) simulator. Comparable results were obtained for RHM, SHV, and CDC. Better performance was obtained when CDC is incorporated with MVT, compared to using CDC and MVT alone. All robust outlier detection algorithms outperformed the standard PCA algorithm. The effect of auto scaling, robust scaling and a new scaling approach called modified scaling were investigated. With the presence of multiple outliers, auto scaling was found to degrade the performance of all the robust techniques. Reasonable results were obtained with the use of robust scaling and modified scaling. [Copyright &y& Elsevier]

Data-driven techniques based on multivariate statistics (such as principal component analysis (PCA) and partial least squares (PLS)) have been applied widely to chemical processes and their effectiveness for fault detection is well recognized. There is an inherent limitation on the ability for purely data-driven techniques to identify and diagnose faults, especially when the abnormal situations are associated with unknown faults or multiple faults. The modified distance (DI) and modified causal dependency (CD) are proposed to incorporate the causal map with data-driven approach to improve the proficiency for identifying and diagnosing faults. The DI is based on the Kullback–Leibner information distance (KLID), the mean of the measured variables, and the range of the measured variable. The DI is used to measure the similarity of the measured variable between the current operating conditions and the historical operating conditions. When the DI is larger than the predefined threshold, the variable is identified as abnormal. The CD, derived based on the multivariate T2 statistic, is used to measure the causal dependency of two variables. When CD is larger than the predefined threshold, the causal dependency of the two variables is broken. The proposed method requires a causal map and historical data associated with the normal operating conditions. A causal map containing the causal relationship between all of the measured variables can be derived based on knowledge from a plant engineer and the sample covariance matrix from the normal data. The DI/CD algorithm outperformed the purely data-driven techniques such as PCA for detecting and identifying known, unknown, and multiple faults using the data sets from the Tennessee Eastman process (TEP). [Copyright &y& Elsevier]

Equipment degradation is ubiquitous in the Chemical Process Industry (CPI), causing significant losses in efficiency, controllability, and plant economy, as well as an increased environmental fingerprint and additional operational safety risks. The case of fouling in heat exchangers, in particular, is well-known and pervasive but still hard to cope with, given the complexity of the underlying mechanisms and the difficulty of assessing its extension in real-time. This problem becomes even more complex in batch processes producing different products, where multiple recipes are used, bringing additional variability and new challenges to the analysis. In this work, we propose a functional data-driven approach for streamlining the analysis and monitoring of the progression of fouling taking place in heat exchangers in multiproduct batch processes. With the approach developed and presented in this paper, process analysis can be efficiently conducted by integrating historical data with engineering knowledge. Furthermore, a surrogate measure of fouling extension in heat exchangers is proposed, that can be readily implemented as an equipment health indicator (EHI) leading to a safer operation of the heat exchanger. [ABSTRACT FROM AUTHOR]

Recent developments in deep learning have brought huge breakthroughs in the image processing area, which triggered numerous successful applications and positively impacted the current big data context of Industry 4.0. On the other hand, it is widely known that large amounts of training data are required to train a deep learning model with millions of parameters, which limits its application in many industrial applications where sufficient data resources are lacking. Transfer learning is one of the practical solutions to reduce the data required for training, which tries to reuse learned knowledge for similar tasks. Nevertheless, many technical details of transfer learning implementation are not well documented. Therefore, in this work, two datasets collected from plastics manufacturing processes were studied to investigate different transfer learning approaches and implementation details for high-performance model building under the constraint of limited available training data. Different transfer learning implementations are compared and important technical details are also discussed in this study. Through this study, the minimum number of training samples can be estimated. Transfer learning is compared with the newly developed few-shot learning approach as a brief comparative study. Finally, this work summarizes practical guidelines for the development of image classification models with limited data resources. • Transfer learning techniques were investigated in this work on industrial datasets. • Implementation details of transfer learning in deep convolutional neural networks including fine-tuning level, model selection etc. were discussed. • The minimum training data size required for image classification was estimated using the transfer learning approach. • A comparison with few-shot learning was studied to evaluate the feasibility of different approaches. • Practical guidelines were summarized for using transfer learning to build image classification models with limited training data. [ABSTRACT FROM AUTHOR]

Summary: This article addresses the plant‐model mismatch estimation problem for linear multiple‐input and multiple‐output systems operating under the dynamic matrix control (DMC) implementation of model predictive control. An autocovariance‐based method is proposed, aiming to identify parameter values that minimize the discrepancy between the theoretical autocovariance matrices derived from implementing the (explicit) DMC control law and the sampled autocovariance matrices calculated from operating data. We provide proof that the method results in unbiased estimates. A means for dealing with potential overfitting issues caused by the finite step response models used in DMC in practice is proposed. Several examples are presented to illustrated the theoretical developments. [ABSTRACT FROM AUTHOR]

Highlights • A new two-stage feature selection methodology for high-dimensional regression problems is proposed. • It is called wide spectrum feature selection for regression (WiSe). • Pearson correlation, Spearman rank correlation, Symmetrical uncertainty, and all their pairwise combinations are considered in the first stage. • In the second stage, the first-pass screened features are further selected using regression-dependent approaches. • Results confirm the increased predictive accuracy of models built after filtering out irrelevant features. Abstract Developing predictive models from industrial datasets implies the consideration of many possible predictor variables (features). Using all available features for data-driven modelling is not recommended, as most of them are expected to be irrelevant and their inclusion in the model may compromise robustness and accuracy. In this work, we present, test and compare a new two-stage feature selection method called wide spectrum feature selection for regression (WiSe). In the first stage, a combination of efficient bivariate filters analyzes linear and non-linear association patterns between predictors and responses, screening out clearly noisy features. In the second stage, the reduced set of retained features is subject to further selection in the scope of the predictive methods considered, optimizing their predictive performance. Three simulated datasets and an industrial case illustrate the effectiveness and benefits of applying WiSe to support model development in a wide range of high-dimensional regression problems. [ABSTRACT FROM AUTHOR]

Manufacturing analytics is of paramount importance in many plants today, and its relevance increases in the current big data context of Industry 4.0. The fields of statistics, chemometrics, and machine learning are expected to provide tools that effectively handle many of the characteristics of industrial data. In this paper, the task of image-based product classification is considered. This is a supervised learning problem where the input is an image and the output is a unique label attributed to the image from a finite set of labels corresponding to the available product classes. This is a prevalent and highly relevant industrial challenge and recent developments in deep learning have proven to be successful in increasing the image classification accuracy, providing state-of-the-art results. Thus, in this work, we leverage deep neural networks' (DNN) ability to automatically learn features from images and test their performance in a real industrial context for predicting the pellet shape. In order to accelerate the training of DNN, transfer learning is employed and a network previously developed for one task is adapted to predict pellet shape. Furthermore, other less complex techniques such as partial least squares discriminant analysis (PLS-DA) and random forests (RF) are also explored in order to assess the benefits of adopting DNN as opposed to current classifiers. An industrial image classification case study was utilized to compare PLS-DA, RF, and DNN models. Compared to the in situ classification system currently in use, increasingly complex models (PLS-DA and RF) were able to better utilize the same pre-defined features and improve prediction accuracy significantly. DNN obtained the highest accuracy on the independent test set, with the advantages of not requiring the a priori computation of image features since they are directly extracted from the raw images. Moreover, by visualizing the output of some layers of the DNN, it is possible to verify that activations occurred in regions that are indeed meaningful for the classification tasks, further supporting that DNN were effectively modelling the relevant features of the pellet. [ABSTRACT FROM AUTHOR]

The area of controller performance monitoring, assessment, and diagnosis for model predictive control (MPC) has seen an increase in interest in recent years. A frequently identified cause of degraded performance is mismatch between the plant model used in the controller and the true dynamics of the plant. Most recent research focuses on locating plant-model mismatch in order to reduce the considerable effort required to re-identify the model. In this paper, we present a novel autocovariance-based plant-model mismatch estimation approach for unconstrained MPC based on linear state space models. We show that (additive) plant-model mismatch can be quantified as the solution of an optimization problem which minimizes the discrepancy between the sample autocovariance of plant outputs and the corresponding value obtained from theoretical predictions. We illustrate our theoretical results with two simulation case studies, demonstrating good performance in estimating parametric mismatch. [ABSTRACT FROM AUTHOR]

We present a data analytics framework for offline analysis of batch processes. The framework provides a unified setting for implementing several variants of feature oriented analysis proposed in the literature, including a new methodology based on the process variable's profiles presented in this article. It also integrates feature generation and feature analysis, in order to speed up the data exploration cycle, which is especially relevant for complex batch processes. The FOBA (Feature Oriented Batch Analytics platform) is described in detail and applied to several case studies regarding different analysis goals: visualization of the differences between the operation of two industrial units (dryers), quality prediction and end-of-batch process monitoring. The performance of the proposed methodology is also critically assessed and compared with other alternative analytical approaches currently in use. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel autocovariance-based plant-model mismatch estimation approach for linear MPC MIMO control loops with changing setpoints and measurable disturbances. Assuming a noise model is available and that there are of periods of operating data where the active set of the controller is fixed and the plant-model mismatch is invariant, we establish an explicit relation between the autocovariance matrices of the mean-centered process outputs and the plant-model mismatch. We then formulate the mismatch estimation problem as an optimization aimed at minimizing the difference between the theoretical autocovariance, computed from the established relation, and actual output autocovariances, calculated from the plant data. We elaborate our results for step-response models typically used in MPC, as well as for parametric (transfer function models) in both continuous time and discrete time. A simulation case study for an unconstrained MPC controller with measurable disturbances is used to illustrate the theoretical results. [ABSTRACT FROM AUTHOR]

In this paper, we present autocovariance-based estimation as a novel methodology for determining plant-model mismatch for multiple-input, multiple-output systems operating under model predictive control. Considering discrete-time, linear time invariant systems under reasonable assumptions, we derive explicit expressions of the autocovariances of the system inputs and outputs as functions of the plant-model mismatch. We then formulate the mismatch estimation problem as a global optimization aimed at minimizing the discrepancy between the theoretical autocovariance estimates and the corresponding values computed from historical closed-loop operating data. Practical considerations related to implementing these ideas are discussed, and the results are illustrated with a chemical process case study. [ABSTRACT FROM AUTHOR]

A significant number of batch process monitoring methods have been proposed since the first groundbreaking approaches were published in the literature, two decades ago. The proper assessment of all the alternatives currently available requires a rigorous and robust assessment framework, in order to assist practitioners in their difficult task of selecting the most adequate approach for the particular situations they face and in the definition of all the optional aspects required, such as the type of preprocessing, infilling, alignment, etc. However, comparison methods currently adopted present several limitations and even some flaws that make the variety of studies available not easily generalizable or, in extreme situations, fundamentally wrong. Without a proper comparison tool decisions are made on a subjective basis and therefore are prone to be at least suboptimal. In this article we present a structured review of comparison methods and figures of merit adopted to assess batch process monitoring approaches, analyzing their strengths and limitations, as well as some common missuses. Furthermore, we propose a comparison and assessment methodology (CAM) and figures of merit that should be adopted in order to circumvent some of the limitations of the current procedures. We begin by addressing, in this article, the analysis of the methods' "detection strength". Detection strength regards the ability to correctly detect abnormal situations without resulting in excessive false alarms. We describe in detail how the comparison framework is implemented and illustrate its application in two case studies, encompassing a rich variety of testing scenarios (99 000 batch runs) and monitoring methods (2-way, 3-way, and dynamic methods, amounting to a total of 60 different combinations of methods and their variants). From this analysis, it stands out that the CAM methodology is suitable for comparing different monitoring methods, even when the number of methods and variants is very large. It provides simple, informative, and statistical-based metrics to assess a given method's performance and is able to quantify the added value of alternative approaches. When applied to the two case studies considered, 2-way methods (with batch-wise unfolding) combined with missing data (MD) infilling and dynamic time warping (DTW) were found to provide adequate solutions. If a more parsimonious model is required, dynamic methods such as autoregressive principal components analysis (ARPCA) provide good alternatives and Tucker3 with current deviations (CD) infilling also presents an interesting performance for some fault scenarios. In general, no method can claim absolute supremacy in all faulty scenarios. [ABSTRACT FROM AUTHOR]

• A statistical learning procedure for Dow data challenge problem (Braun et al., 2020) is presented that integrates process knowledge in all steps from pre-processing and model interpretation. • An accurate inferential sensor model is built with online bias learning based on new data to predict the impurity in the product stream with apparent drifts. • Least angle regression solution (LARS) is shown to select only one variable among a set of collinear variables. • We report the detection of an equipment-switching operation in the data and interpolations found in the impurity data, which leads to unique data pre-processing measures. • Using a softplus function, we propose a method to deal with non-negative physical property modeling. In this paper, we propose a statistical learning procedure that integrates process knowledge for the Dow data challenge problem presented in Braun et al. (2020). The task is to build an accurate inferential sensor model to predict the impurity in the product stream with apparent drifts. The proposed method consists of i) process data exploratory analysis, ii) a method for variable selection, iii) a method to deal with non-negative physical property modeling using a softplus function; and iv) a method for online bias updating based on known data. We make use of process operation knowledge in all steps of data analytics, including exploratory analysis and feature selection. We report the detection of equipment-switching operations in the data and interpolations found in the impurity data. Partial least squares (PLS) and least angle regression solution (LARS) are adopted to model the data with strong collinearity. Pros and cons of LARS and PLS are given with practical implications. [ABSTRACT FROM AUTHOR]

• Hybrid modeling has been attracting the interest of the scientific community for almost 30 years. • Big data and the industry 4.0 bring opportunities for new hybrid modeling solutions. • We review hybrid modeling schemes, their training, validation and applications. • Usually mechanistic models are improved by data-driven models. • There is the need for a generic framework balancing prior and data-driven knowledge. The chemical processing industry has relied on modeling techniques for process monitoring, control, diagnosis, optimization, and design, especially since the third industrial revolution and the emergence of Process Systems Engineering. The fourth industrial revolution, connected to massive digitization, made it possible to collect and process large volumes of data triggering the development of data-driven frameworks for knowledge extraction. However, one must not leave behind the successful solutions developed over decades based on first principle mechanistic modeling approaches. At present, both industry and researchers are realizing the need for new ways to incorporate process and phenomenological knowledge in big data and machine learning frameworks, leading to more robust and intelligible artificial intelligence solutions, capable of assisting the target stakeholders in their activities and decision processes. In this article, we review hybrid modeling techniques, associated system identification methodologies and model assessment criteria. Applications in chemical and biochemical processes are also referred. [ABSTRACT FROM AUTHOR]

This paper explores dimensionality reduction (DR) approaches for visualizing high dimensional data in chemical processes. Visualization provides powerful insight and process understanding in the industrial context, and accelerates process troubleshooting. A diverse array of existing, easy-to-use DR methods are evaluated in three case studies on large-scale industrial manufacturing plants. Supervised and unsupervised cases are presented with the objective of solving typical industrial problems related to unplanned events, plant performance improvement, and quality underperformance troubleshooting. For the unsupervised case, the evaluation aims to identify approaches that provide insight beyond those of PCA (Principal Component Analysis), and also examines quality metrics of the reduced (latent) space which characterize the degree of trust in the DR. UMAP (Uniform Manifold Approximation and Projection) outperforms other techniques, bringing new insights when comparing with other methods. For the supervised case, UMAP is combined with traditional variable selection methods, such as VIP (Variable Influence on Projection) weights from PLS-DA (Partial Least Squares Discriminant Analysis), in order to improve latent space visualization by increasing separation between classes. • Dimensionality reduction techniques are applied to three industrial case studies. • UMAP has outperformed other methods that solve unsupervised problems. • Combining UMAP with feature selection techniques improves latent space visualization. [ABSTRACT FROM AUTHOR]