Search

Smart healthcare is a framework that utilizes technologies such as wearable devices, the Internet of Medical Things (IoMT), sophisticated machine learning algorithms, and wireless communication technology to seamlessly access health records, link individuals, resources, and organizations, and then effectively handle and react to health environment demands intelligently. One of the main ingredients of smart healthcare is medical sensors or IoMT. Due to the complex nature of diseases, in many cases, there is a need for multimodal medical signals for their diagnoses. While using multimodal signals, the most important issue is how to fuse them – an area of burgeoning interest within the research community. This paper presents a comprehensive survey of multimodal medical signals fusion schemes that have been proposed for smart healthcare applications. Research works included in major repositories, such as IEEE Xplore, Science Direct, Springer Link, and ACM digital library have been surveyed to address several related research questions. Focus is placed on recent developments, thus only works published between 2014-2020 are considered. Finally, key research challenges and possible future directions are also provided.

The significant increase in the number of individuals with chronic ailments (including the elderly and disabled) has dictated an urgent need for an innovative model for healthcare systems. The evolved model will be more personalized and less reliant on traditional brick-and-mortar healthcare institutions such as hospitals, nursing homes, and long-term healthcare centers. The smart healthcare system is a topic of recently growing interest and has become increasingly required due to major developments in modern technologies, especially in artificial intelligence (AI) and machine learning (ML). This paper is aimed to discuss the current state-of-the-art smart healthcare systems highlighting major areas like wearable and smartphone devices for health monitoring, machine learning for disease diagnosis, and the assistive frameworks, including social robots developed for the ambient assisted living environment. Additionally, the paper demonstrates software integration architectures that are very significant to create smart healthcare systems, integrating seamlessly the benefit of data analytics and other tools of AI. The explained developed systems focus on several facets: the contribution of each developed framework, the detailed working procedure, the performance as outcomes, and the comparative merits and limitations. The current research challenges with potential future directions are addressed to highlight the drawbacks of existing systems and the possible methods to introduce novel frameworks, respectively. This review aims at providing comprehensive insights into the recent developments of smart healthcare systems to equip experts to contribute to the field.

Driver inattention is among major contributing factors to traffic accidents. There have been and continue to be efforts by governing bodies, car manufacturers, and researchers to prevent driver inattention or, failing that, to mitigate its effects. Many vehicles nowadays come equipped with driver monitoring systems that can alert the driver to, or compensate for, inattention. Moreover, the research community continues to explore and investigate more robust approaches to deal with inattention. Meanwhile, vehicle automation, to various degrees, is becoming more prevalent, with the human’s role in the driving task changing depending on the level of autonomy. This necessitates that inattention detection, moving forward, be studied and designed in view of automation and in the context of a specific level of vehicle autonomy. Driver inattention and vehicle automation interact in a complex way, and that needs to be taken into account in the design of future vehicles. We explore this interaction in this paper in light of research findings, and survey inattention detection systems and attempt to contextualize them within popular frameworks for next-generation autonomous vehicles.

Distracted driving is among the primary causes for serious car accidents. Among the leading cause of death among teenagers today are traffic accidents and major part of them are related to distracted driving. We propose here an end-to-end Convolutional Neural Network-based driver distraction recognition (DDR) system that can generalize to diverse driving conditions. The proposed method consists of two steps: developing generative models to produce images of different driving scenarios and developing a discriminative model for image classification. Unlike traditional methods based on image data-sets collected by simulation experiments, we collect a diverse data-set of drivers in different driving conditions and activity patterns from the Internet and train generative models for multiple driving scenarios. By sampling from these generative models, we augment the collected data-set with new training samples and train a Convolutional Neural Network for distraction recognition. We demonstrate that the generative models are able to generate images of drivers in different driving scenarios. With augmentative images, the DDR system achieves an improvement of 11.45% on image classification performance in a driving simulation environment. Moreover, we demonstrate how the trained DDR systems can be integrated within a driver monitoring system.

Many of today's vehicles come equipped with Advanced Driver Assistance Systems (ADAS). Proactive ADAS have the ability to predict short term driving situations. This provides drivers more time to take adequate actions to avoid or mitigate driving risks. In this work, we address the question of predicting drivers’ imminent maneuvers before they perform an actual steering operation. The proposed system uses deep recurrent neural networks to fuse the information regarding driver observation actions and the driving environment. With new data labeling methods and effective sequential modeling approaches, the system is able to predict with high accuracy driving maneuvers shortly before the actual steering operations. A set of experiments show that the proposed approach anticipates lane change maneuvers 1.50 seconds before cars start to yaw with an accuracy improved to 90.52% and anticipates turn maneuvers at intersections with green lights 2.53 seconds before cars start to yaw with an accuracy improved to 78.59%. We also show in this work how the system can be adapted for driving proficiency assessment.

We propose a new embedding method, named Quantile-Quantile Embedding (QQE), for distribution transformation and manifold embedding with the ability to choose the embedding distribution. QQE, which uses the concept of quantile-quantile plot from visual statistical tests, can transform the distribution of data to any theoretical desired distribution or empirical reference sample. Moreover, QQE gives the user a choice of embedding distribution in embedding the manifold of data into the low dimensional embedding space. It can also be used for modifying the embedding distribution of other dimensionality reduction methods, such as PCA, t-SNE, and deep metric learning, for better representation or visualization of data. We propose QQE in both unsupervised and supervised forms. QQE can also transform a distribution to either an exact reference distribution or its shape. We show that QQE allows for better discrimination of classes in some cases. Our experiments on different synthetic and image datasets show the effectiveness of the proposed embedding method., Comment: Published in Machine Learning with Applications, Elsevier, Volume 6, Pages 100088, 2021

Convolutional Recurrent architectures are currently preferred for spatio-temporal learning tasks in videos to the 3D convolutional networks which accompany a huge computational burden and it is imperative to understand the working of different architectural configurations. But most of the current works on visual learning, especially for video anomaly detection, predominantly employ ConvLSTM networks and focus less on other possible variants of Convolutional Recurrent configurations for temporal learning which warrants a need to study the different possible variants to make informed, optimal design choices according to the nature of the application at hand. We explore a variety of Convolutional Recurrent architectures and the influence of hyper-parameters on their performance for the task of anomaly detection. Through this work, we also intend to quantify the efficiency of the architectures based on the trade-off between their performance and computational complexity. With comprehensive quantitative and visual evidence, we establish that the ConvGRU based configurations are the most effective and perform better than the popular ConvLSTM configurations on video anomaly detection tasks, in contrast to what is seen from the literature.

Since their inception, AutoEncoders have been very important in representational learning. They have achieved ground-breaking results in the realm of automated unsupervised anomaly detection for various critical applications. However, anomaly detection through AutoEncoders suffers from lack of transparency when it comes to decision making based on the outputs of the AutoEncoder network, especially for image-based models. Though the residual reconstruction error map from the AutoEncoder helps explaining anomalies to a certain extent, it is not a good indicator of the implicitly learnt attributes by the model. A human interpretable explanation of why an instance is anomalous not only enables the experts to fine-tune the model but also establishes and increases trust by non-expert users of the model. Convolutional AutoEncoders in particular suffer the most as there are only limited studies that focus on transparency and explainability. In this paper, aiming to bridge this gap, we explore the feasibility and compare the performances of several State-of-the-Art Explainable Artificial Intelligence (XAI) frameworks on Convolutional AutoEncoders. The paper also aims at providing the basis for future developments of reliable and trustworthy AutoEncoders for visual anomaly detection.

This paper presents a panel summary on the framework of Autonomous Systems (AS) and paradigms in development. AS are advanced intelligent systems and general AI technologies triggered by the transdisciplinary development in intelligence science, system science, brain science, cognitive science, robotics, computational intelligence, and intelligent mathematics. It is recognized that, in a rigorous perspective, the only matured AS is human brains and human collective intelligence. It explains why there was rarely man-made AS in the past half century, because of the theoretical, mathematical, computational, and programming language unreadiness. Therefore, the ultimate goal of AS is to implement a brain-inspired system that may think and behave as a human counterpart in hybrid intelligent systems and general AI implementations. There is no doubt that AS will be increasingly demanded by the intelligence-based industries and societies for cognitive computers, deep machine learning systems, robotics, brain-inspired systems, mission-critical systems, self-driving vehicles, and intelligent appliances.

Understanding the notion of normality in visual data is a complex issue in computer vision with plenty of potential applications in several sectors. The immense effort required for optimal design for real-world application of existing methods warrants the need for a generic framework that is efficient, automated and can be momentarily deployed for the operation, reducing the effort expended on model design and hyper-parameter tuning. Hence, we propose a novel, modular and model-agnostic improvement to the conventional AutoEncoder architecture, based on visual soft-attention for the inputs to make them robust and readily improve their performance in automated semi-supervised visual anomaly detection tasks, without any extra effort in terms of hyperparameter tuning. Besides, we discuss the role of attention in AutoEncoders (AE) that can significantly improve learning and the efficacy of the models with detailed experimental results on diverse visual anomaly detection datasets.

The ability to predict crop yield is vital for food security worldwide and forecasting crop prices can help farmers avoid price crash. In this work, an investigation of using satellite images and deep learning models to predict crop yields as well as forecasting farmers' prices is conducted. For tractability, dimensionality reduction is achieved by converting the images to histograms representing the pixel frequency. The models tested are LSTM, CNN, CNN-LSTM, CNN-LSTM ensemble as well as a Gaussian Process added to each for enhanced performance. It is found that the proposed ensemble of CNN-LSTMs is the best in predicting the yearly soybean yields in addition to forecasting the daily strawberry yields and prices. It outperforms models suggested in the literature with an improvement of 31% in terms of average Root Mean Square Error (RMSE).

To overcome the computational complexity of retraining Deep Learning (DL) yield forecasting models for each type of Fresh Produce (FP), it is necessary to have a generalization of the models' application to similar FP. This can be done by transferring the learning among similar FP with minimal retraining. Hence, Transfer Learning (TL) is used in this work amongst berries which are similar in nature. First, the proposed DL model is trained using station-based data and satellite images as inputs mapped to the strawberry yield as output. The weights obtained from this learning are transferred to the raspberry yield forecasting model since raspberry and strawberry yields are similar and are both planted in California. The proposed model is an ensemble of two models: the station-based ensemble model (ATT-CNN-LSTM-SeriesNet_Ens) with its compound DL components, SeriesNet with Gated Recurrent Unit (GRU) and Convolutional Neural Network LSTM with Attention layer (Att-CNN-LSTM); trained and tested using station-based data as input and the corresponding strawberry yields as output. Second, the remote sensing ensemble model (SIM_CNN-LSTM_Ens), which is an ensemble of Convolutional Neural Network LSTM (CNN-LSTM) models; trained and tested using satellite images as input mapped to the same yields as output. The weights obtained are transferred to the raspberry yield forecasting ensemble model with minimal retraining. It is found that the voting ensemble improves performance by 27% compared to the best performing component model. Based on an aggregated measure, the performance obtained from TL is comparable to that obtained by training the models on the raspberry data without TL, while having around 55 % reduction in processing time.

It is common for a time series dataset to have missing values, and it is necessary to fill these missing elements before using the dataset for training forecasting models. Usually this problem is tackled using non-machine learning methods that introduce bias into the system which results in unreliable forecasting results. Moreover, most of the work found in the literature tackles imputation of missing values when they are randomly scattered in the dataset while very little work is found tackling the case of consecutive occurrence of missing data; i.e. missing data chunks in the dataset. Therefore, in this work, comprehensive imputation models are developed to impute both random as well as chunks of missing values. Alongside, a framework is found enabling the user to impute any time series data with the optimal models. In order to carry out the task, one non-deep machine learning model (Bidirectional Imputation model) and three deep learning (DL) imputation models (Ensemble model, Transfer Learning model and Hybrid model), are tested using complete time series. The results show that the hybrid model yields a maximum of 38% improvement in the Aggregate Error (AGE) when compared with other models.

Histopathology image embedding is an active research area in computer vision. Most of the embedding models exclusively concentrate on a specific magnification level. However, a useful task in histopathology embedding is to train an embedding space regardless of the magnification level. Two main approaches for tackling this goal are domain adaptation and domain generalization, where the target magnification levels may or may not be introduced to the model in training, respectively. Although magnification adaptation is a well-studied topic in the literature, this paper, to the best of our knowledge, is the first work on magnification generalization for histopathology image embedding. We use an episodic trainable domain generalization technique for magnification generalization, namely Model Agnostic Learning of Semantic Features (MASF), which works based on the Model Agnostic Meta-Learning (MAML) concept. Our experimental results on a breast cancer histopathology dataset with four different magnification levels show the proposed method's effectiveness for magnification generalization., Accepted for presentation at International Symposium on Biomedical Imaging (ISBI'2021)

Variants of Triplet networks are robust entities for learning a discriminative embedding subspace. There exist different triplet mining approaches for selecting the most suitable training triplets. Some of these mining methods rely on the extreme distances between instances, and some others make use of sampling. However, sampling from stochastic distributions of data rather than sampling merely from the existing embedding instances can provide more discriminative information. In this work, we sample triplets from distributions of data rather than from existing instances. We consider a multivariate normal distribution for the embedding of each class. Using Bayesian updating and conjugate priors, we update the distributions of classes dynamically by receiving the new mini-batches of training data. The proposed triplet mining with Bayesian updating can be used with any triplet-based loss function, e.g., triplet-loss or Neighborhood Component Analysis (NCA) loss. Accordingly, Our triplet mining approaches are called Bayesian Updating Triplet (BUT) and Bayesian Updating NCA (BUNCA), depending on which loss function is being used. Experimental results on two public datasets, namely MNIST and histopathology colorectal cancer (CRC), substantiate the effectiveness of the proposed triplet mining method., Accepted for presentation at the 25th International Conference on Pattern Recognition (ICPR), IEEE, 2020. The first two authors contributed equally to this work

The importance of imputation for having highly performing prediction models is highlighted in this work. Three imputation techniques are tested against a non-imputation approach that discards records with any missing values; the complete-case analysis (CCA). The deep learning linear memory vector recurrent neural network-RNN (LIME) imputation model is tested along with two other nondeep learning models such as the linear function and Last Observation Carried Forward (LOCF). The simple LSTM deep learning (DL) prediction model is deployed to decide the best performing imputation model, the one resulting in the lowest price and yield prediction errors. Five performance evaluation measures are utilized; the mean absolute error (MAE), the root mean square error (RMSE), R2 correlation measure along with two aggregated measures summarizing these three measures to decide the overall prediction performance; the average aggregated measure (AGM) for each considered step ahead and the average of the AGM across all considered steps ahead (AAGM). Based on AGM, it is found that the LIME imputation model leads to the best prediction performance of the simple LSTM DL model across both applications of 5 weeks ahead strawberry price and yield predictions using weather; W2P and W2Y. Therefore, the LIME imputed file is reused to train two compound DL models, Convolutional Long Short-Term Memory RNN with attention (ATT-ConvLSTM) and ATT-CNN-LSTM along with their Voting Regressor ensemble (VR). The same models are retrained with files preprocessed with the non-imputation approach, CCA. It is found that the overall AAGM of the compound DL and ensemble prediction models across all the 1, 2, 3, and 4 weeks ahead price predictions confirm that using LIME highly improves the prediction performance of the ensemble and its compound DL components. The VR ensemble price prediction performance is improved by 72% and the ATTConvLSTM component is improved by 89% compared to their performances without imputation; using CCA preprocessed files.

Missing data are commonly found in time series datasets. These missing elements are usually a hurdle in utilizing the datasets in prediction or forecasting, making imputation of those missing values imperative. Due to the non-linear dependencies between the current and previous values, imputation remains a challenging task. Conventional methods such as averaging, deletion or filling with the last observed value add bias to the data and are therefore inefficient. Since different time series showcase varying characteristics, figuring out which imputation method works best for the respective time series is essential. In this work, seven different deep learning (DL) imputation methods are examined along with three machine learning (ML) ensembles. To enable a recommendation of the best imputation method for each time series type, the imputation models are tested using the four main types of time series: trend (T), seasonal (S), combined trend and seasonal time series (T&S) and random (R) time series. Results indicate that the Gated Recurrent Unit (GRU) neural networks are, in general, the best for missing values imputation with varying complexity based on the time series type. For example, it is found that the residual GRU is recommended for the trend and seasonal time series while the GRU is recommended for the combined type. Conversely, all tested DL imputation models can be used with the random time series type. In addition, the considered ML ensembles do not perform as high as the DL models with all tested types of times series.

Time series prediction has been challenging topic in several application domains. In this paper, an ensemble of two top performing deep learning architectures across different applications such as fresh produce (FP) yield prediction, FP price prediction and crude oil price prediction is proposed. First, the input data is trained on an array of different machine learning architectures, the top two performers are then combined using a stacking ensemble. The top two performers across the three tested applications are found to be Attention CNN-LSTM (AC-LSTM) and Attention ConvLSTM (ACV-LSTM). Different ensemble techniques, mean prediction, Linear Regression (LR) and Support vector Regression (SVR), are then utilized to come up with the best prediction. An aggregated measure that combines the results of mean absolute error (MAE), mean squared error (MSE) and R2 coefficient of determination (R2) is used to evaluate model performance. The experiment results show that across the various examined applications, the proposed model which is a stacking ensemble of the AC-LSTM and ACV-LSTM using a linear SVR is the best performing based on the aggregated measure.

Adequately priced orders and time for fresh produce (FP) are two factors that bring commercial benefits to vendors and minimizes waste. However, many factors, such as income, labor, and other trade issues, affect the price that include uncertainties due to climate change, making decisions on FP procurement prices and quantities extremely challenging. Two artificial intelligence-based forecasting tools, i.e., a single variate and a multivariate model, are trained, tested, and compared in this study to predict future daily offer prices up to 7 days ahead for strawberries using mutual transactions for the distribution centers of Loblaws Companies Limited (LCL) in Canada. Results reveal that the developed multivariate model, utilizing both prices of the LCL dataset and California's strawberries yield dataset as predictors, outperforms the best single variate model.

There has been a tremendous increase in the costs of caring for older adults owing to the fact that societies are aging around around the world. This has led to a decrease in the number of caregivers who are able to assist. Investigative studies indicate that older adults require social as well as physical support for their well-being which prompted researchers to use social and cognitive robots and advanced human machine interaction devices. However, most of these studies have shortcomings when it comes to providing means of a natural interaction with the machine. With speech being the most natural way for human communication and the huge developments in the Internet of Things and smart homes, equipping a robotic system with powerful natural speech interaction capabilities to maintain a conversation with an elderly while being linked to other smart home devices shows a promising direction. This paper describes a scalable and expandable system with main goal of designing a natural speech-enabled system for older adults that is capable of linking to multiple active agents with minimal integration efforts. The system makes use of the power of commercially available digital assistant systems, integrated with an intelligent conversational agent, robotics, and smart wearables. The main advantage of the system is that it could provide a portion of the population, namely older adults and the disabled, the flexibility of interacting naturally with powerful social robots in smart home environments, hence providing them with much needed independence.

It is common for a time series dataset to have missing values, and it is necessary to fill these missing elements before fitting any model for forecasting or prediction. Time series imputation remains a challenging task due to the existence of non-linear dependencies between current and past values. Conventional methods, such as deletion of rows containing missing values or filling them with the last observed value, add bias to the data and are therefore inefficient. There are situations where data is missing at consecutive points or random points in the dataset, and one particular method may not work well for all cases. In this paper, nine commonly used models in the field of imputation, based on tools of statistics, machine learning, and deep learning, are compared. Results show that Linear Memory Vector Gated Recurrent Unit (LIME-GRU) outperforms the other tested models by having the least Mean Square Error (MSE) and Root Mean Squared Error (RMSE). A predictive model to gauge the impact of imputation on prediction is also used to validate the findings. The results of the prediction model illustrate that with LIME-GRU, there was a 39% improvement in Average Aggregated Measure (AAGM) when compared with mode imputation on a particular test case.

Agent-based modeling has been proposed to simulate real world situations where autonomous agents take their own decisions based on simple rules and data from the environment. The strawberry market in California is a challenging example as prices can vary suddenly due to change in supplies and the fruits cannot be stored for long durations. The microeconomic theory that is expected to model this market is implemented within the simulation model to predict the strawberry price based on the difference between total supply and total demand. In this study, the observed yield of strawberry for the two main suppliers in California is considered as total supply; and for predicting the demand, different demand functions are presented. To estimate the ABM model parameters, two optimization methods are applied with Python-Netlogo. Finally, the computational results are presented to show the performance of the prediction model with directions for future research for improving the results.

Metric learning is one of the techniques in manifold learning with the goal of finding a projection subspace for increasing and decreasing the inter- and intra-class variances, respectively. Some of the metric learning methods are based on triplet learning with anchor-positive-negative triplets. Large margin metric learning for nearest neighbor classification is one of the fundamental methods to do this. Recently, Siamese networks have been introduced with the triplet loss. Many triplet mining methods have been developed for Siamese networks; however, these techniques have not been applied on the triplets of large margin metric learning for nearest neighbor classification. In this work, inspired by the mining methods for Siamese networks, we propose several triplet mining techniques for large margin metric learning. Moreover, a hierarchical approach is proposed, for acceleration and scalability of optimization, where triplets are selected by stratified sampling in hierarchical hyper-spheres. We analyze the proposed methods on three publicly available datasets, i.e., Fisher Iris, ORL faces, and MNIST datasets., The first two authors contributed equally to this work

The currently deployed prediction models for strawberry fresh produce (FP) are based on either conventional machine learning (ML) or on simple deep learning (DL) models that are mostly applied for yield prediction. In this paper, we propose more comprehensive DL models that are applied for the first time to predict strawberry yield. The strawberry price is predicted as well directly from weather input parameters and yield. The strawberry price prediction is achieved using compound DL models such as Convolutional Long Short-Term Memory Recurrent Neural Network (CNN-LSTM). It is found that by adding attention, the performance of the compound models usually improves. After utilizing an aggregated performance measure to find the best model, the Attention-CNN-LSTM model proved to be the best compared to the rest of the deployed conventional ML models as well as the compound and simple DL models. The aggregated measure shows that this model is capable of precisely predicting strawberry prices five weeks ahead while maintaining the lowest prediction error and the highest model correlation.

Building highly precise prediction models for Fresh Produce (FP) market price is crucial to protect retailers from overpriced FP. In this paper we are comparing the price prediction models performance of deep learning (DL) models with statistical as well as standard machine learning (ML) models. Five types of FP are considered in performance testing. It is found that the conventional ML models outperform the statistical models such as ARIMA. On the other hand, the winning model among the conventional ML models (the Gradient Boosting model) proves to be less performant as compared with the simple or compound DL models. Moreover, the simple DL models, such as the Long Short-Term Memory (LSTM), are outperformed by the compound one, the Convolutional Long Short-Term Memory Recurrent Neural Network (CNN-LSTM), whose performance improves by adding attention. The model is capable of precisely predicting FP prices for up to three weeks ahead.

We present a new method which generalizes subspace learning based on eigenvalue and generalized eigenvalue problems. This method, Roweis Discriminant Analysis (RDA) named after Sam Roweis, is a family of infinite number of algorithms where Principal Component Analysis (PCA), Supervised PCA (SPCA), and Fisher Discriminant Analysis (FDA) are special cases. One of the extreme special cases, named Double Supervised Discriminant Analysis (DSDA), uses the labels twice and it is novel. We propose a dual for RDA for some special cases. We also propose kernel RDA, generalizing kernel PCA, kernel SPCA, and kernel FDA, using both dual RDA and representation theory. Our theoretical analysis explains previously known facts such as why SPCA can use regression but FDA cannot, why PCA and SPCA have duals but FDA does not, why kernel PCA and kernel SPCA use kernel trick but kernel FDA does not, and why PCA is the best linear method for reconstruction. Roweisfaces and kernel Roweisfaces are also proposed generalizing eigenfaces, Fisherfaces, supervised eigenfaces, and their kernel variants. We also report experiments showing the effectiveness of RDA and kernel RDA on some benchmark datasets.

Fisher Discriminant Analysis (FDA) is a subspace learning method which minimizes and maximizes the intra- and inter-class scatters of data, respectively. Although, in FDA, all the pairs of classes are treated the same way, some classes are closer than the others. Weighted FDA assigns weights to the pairs of classes to address this shortcoming of FDA. In this paper, we propose a cosine-weighted FDA as well as an automatically weighted FDA in which weights are found automatically. We also propose a weighted FDA in the feature space to establish a weighted kernel FDA for both existing and newly proposed weights. Our experiments on the ORL face recognition dataset show the effectiveness of the proposed weighting schemes.

Anomaly detection is a pattern recognition task that aims at distinguishing abnormal patterns from normal ones. In this paper, we propose a convolutional auto-encoder based model to detect anomaly images by producing a sparse representation in the latent space. The proposed approach is able to represent the normal images using sparse encoding and the encoding can be well reconstructed by the decoder. However, the learned convolutional filters are not able to represent the abnormal images in a sparse way. Therefore, the decoder can not reconstruct the abnormal images with high quality. By assessing the reconstruction performance, we can distinguish the abnormal images from the normal ones. The experimental results show the superiority of our proposed model over other variants of auto-encoder based anomaly detection models in terms of AUC. In addition, the results show that the sparse representation based anomaly detection method could apply to different scenarios.

Novel coronavirus (COVID-19) outbreak, has raised a calamitous situation all over the world and has become one of the most acute and severe ailments in the past hundred years. The prevalence rate of COVID-19 is rapidly rising every day throughout the globe. Although no vaccines for this pandemic have been discovered yet, deep learning techniques proved themselves to be a powerful tool in the arsenal used by clinicians for the automatic diagnosis of COVID-19. This paper aims to overview the recently developed systems based on deep learning techniques using different medical imaging modalities like Computer Tomography (CT) and X-ray. This review specifically discusses the systems developed for COVID-19 diagnosis using deep learning techniques and provides insights on well-known data sets used to train these networks. It also highlights the data partitioning techniques and various performance measures developed by researchers in this field. A taxonomy is drawn to categorize the recent works for proper insight. Finally, we conclude by addressing the challenges associated with the use of deep learning methods for COVID-19 detection and probable future trends in this research area. This paper is intended to provide experts (medical or otherwise) and technicians with new insights into the ways deep learning techniques are used in this regard and how they potentially further works in combatting the outbreak of COVID-19., Comment: 18 pages, 2 figures, 4 Tables

Siamese neural network is a very powerful architecture for both feature extraction and metric learning. It usually consists of several networks that share weights. The Siamese concept is topology-agnostic and can use any neural network as its backbone. The two most popular loss functions for training these networks are the triplet and contrastive loss functions. In this paper, we propose two novel loss functions, named Fisher Discriminant Triplet (FDT) and Fisher Discriminant Contrastive (FDC). The former uses anchor-neighbor-distant triplets while the latter utilizes pairs of anchor-neighbor and anchor-distant samples. The FDT and FDC loss functions are designed based on the statistical formulation of the Fisher Discriminant Analysis (FDA), which is a linear subspace learning method. Our experiments on the MNIST and two challenging and publicly available histopathology datasets show the effectiveness of the proposed loss functions., Comment: Accepted (to appear) in International Joint Conference on Neural Networks (IJCNN) 2020, IEEE, in IEEE World Congress on Computational Intelligence (WCCI) 2020

After the tremendous development of neural networks trained by backpropagation, it is a good time to develop other algorithms for training neural networks to gain more insights into networks. In this paper, we propose a new algorithm for training feedforward neural networks which is fairly faster than backpropagation. This method is based on projection and reconstruction where, at every layer, the projected data and reconstructed labels are forced to be similar and the weights are tuned accordingly layer by layer. The proposed algorithm can be used for both input and feature spaces, named as backprojection and kernel backprojection, respectively. This algorithm gives an insight to networks with a projection-based perspective. The experiments on synthetic datasets show the effectiveness of the proposed method.

A decade ago, the crowdsourcing term was first coined and used to represent a method for expressing the wisdom of the crowd in accomplishing two types of tasks. One type includes tasks that need human intelligence rather than machines, and the other type covers those tasks that can be accomplished with a higher time and cost efficiency using the crowd rather than employing experts. The crowdsourcing process contains five modules: The first is designing incentives to mobilize the crowd to do the required task. This step is followed by four modules for collecting and assuring quality and then verifying and aggregating the received information. The verification and quality control can be done for the tasks, collected data and the participants by having more participants answer the same question or accepting answers only from experts to avoid errors from unreliable participants. Methods of discovering topic experts are utilized to discover reliable candidates in the crowd who have relevant experience in the discussed topic. Expert discovery reduces the number of needed participants per question which reduces the overall cost. This work summarizes and reviews the methods used to accomplish each processing step. Yet, choosing a specific method remains application dependent.

Data often have nonlinear patterns in machine learning. One can unfold the nonlinear manifold of a dataset for low-dimensional visualization and feature extraction. Locally Linear Embedding (LLE) is a nonlinear spectral method for dimensionality reduction and manifold unfolding. It embeds data using the same linear reconstruction weights as in the input space. In this paper, we propose an open source module which not only implements LLE, but also includes implementations of two generative LLE algorithms whose linear reconstruction phases are stochastic. Using this module, one can generate as many manifold unfoldings as desired for data visualization or feature extraction.

Detecting topics from Twitter streams has become an important task as it is used in various fields including natural disaster warning, users opinion assessment, and traffic prediction. In this article, we outline different types of topic detection techniques and evaluate their performance. We categorize the topic detection techniques into five categories which are clustering, frequent pattern mining, Exemplar-based, matrix factorization, and probabilistic models. For clustering techniques, we discuss and evaluate nine different techniques which are sequential k-means, spherical k-means, Kernel k-means, scalable Kernel k-means, incremental batch k-means, DBSCAN, spectral clustering, document pivot clustering, and Bngram. Moreover, for matrix factorization techniques, we analyze five different techniques which are sequential Latent Semantic Indexing (LSI), stochastic LSI, Alternating Least Squares (ALS), Rank-one Downdate (R1D), and Column Subset Selection (CSS). Additionally, we evaluate several other techniques in the frequent pattern mining, Exemplar-based, and probabilistic model categories. Results on three Twitter datasets show that Soft Frequent Pattern Mining (SFM) and Bngram achieve the best term precision, while CSS achieves the best term recall and topic recall in most of the cases. Moreover, Exemplar-based topic detection obtains a good balance between the term recall and term precision, while achieving a good topic recall and running time.

Although the future impact of plug-in electric vehicles (PEVs) on distribution grids is disputed, all parties agree that mass operation of PEVs will greatly affect load profiles and grid assets. The large-scale penetration of domestic energy storage, such as with photovoltaics (PVs), into the edges of low-voltage grids is increasing the amount of customer-generated electricity. Distribution grids, which are inherently unbalanced, tend to become even more so with the uneven spread of PVs and PEVs. In combination, PEVs and local generation could provide voltage support for distribution networks, and support increased penetration. This paper develops an interactive energy management system for incorporating PEVs in demand response (DR). Using this system, owners can immediately choose whether they want to discharge their PEV battery back into the grid. The system not only provides owners with a flexible scheme for contributing to DR but also ensures that, through real-time collaboration of PEVs and PVs, the three-phase grid operates within acceptable voltage unbalance. An extensive performance evaluation using MATLAB/GAMS simulation of the 123-bus test system verifies the effectiveness of the proposed approach.

Cellular automata (CA), widely used in modeling and simulation of complex systems, have recently been utilized as an optimizer. This paper presents an application of CA to the optimization of a real-world renewable hydropower energy system operations, i.e. Karoon2-Karoon3 cascade water reservoirs and their powerplants in Iran. The problem is formulated as a nonlinear nonconvex mathematical program in which CA are used to optimize long-term multi-stage operations and scheduling of the reservoirs and the powerplants in terms of the amounts of the reservoirs storage volumes and flows passing through the turbines in every time step. Comparison of CA results with those obtained by classical gradient-based nonlinear programming techniques demonstrate that the CA-based optimizer performs quite satisfactorily in terms of converging speed and quality of its solutions.

Breast cancer is a current problem that causes the death of many women. In this work, we test meta-heuristics applied to the segmentation of mammographic images. Traditionally, the application of these algorithms has a direct relationship with optimization problems, however, in this study, its implementation is oriented to the segmentation of mammograms using the Dunn index as an optimization function, and the grey levels to represent each individual. The update of grey levels during the process results in the maximization of the Dunn&rsquo, s index function, the higher the index, the better the segmentation will be. The results showed a lower error rate using these meta-heuristics for segmentation compared to a well-adopted classical approach known as the Otsu method.

Neural networks trained on tasks sequentially tend to degrade in performance, on the average, the more tasks they see, as the representations learned for one task get progressively modified while learning subsequent tasks. This phenomenon— known as catastrophic forgetting—is a major obstacle on the road toward designing agents that can continually learn new concepts and tasks the way, say, humans do. A common approach to containing catastrophic forgetting is to use regularization to slow down learning on weights deemed important to previously learned tasks. We argue in this paper that, on their own, such post hoc measures to safeguard what has been learned can, even in their more sophisticated variants, paralyze the network and degrade its capacity to learn and counter forgetting as the number of tasks learned increases. We propose instead— or possibly in conjunction—that, in anticipation of future tasks, regularization be applied to drive the optimization of network weights toward reusable solutions. We show that one way to achieve this is through an auxiliary unsupervised reconstruction loss that encourages the learned representations not only to be useful for solving, say, the current classification task, but also to reflect the content of the data being processed—content that is generally richer than it is discriminative for any one task. We compare our approach to the recent elastic weight consolidation regularization approach, and show that, although we do not explicitly try to preserve important weights or pass on any information about the data distribution of learned tasks, our model is comparable in performance, and in some cases better.

Catastrophic forgetting has long been seen as the main obstacle to building continual learning models. We argue in this paper that an equally challenging characteristic of the continual learning framework is that data are never completely available at the same time, making it difficult to learn joint conditional distributions over them. This is most evident in the usually large gap between single-head and multi-head performance of continual learning models. We propose in this paper two strategies to improve performance of continual learning models, particularly in the single-head framework and for image classification tasks. First, we argue that learning multiple binary classifiers, rather than a single multi-class classifier, for each presentation of data is more consistent with the single-head framework. Moreover, we argue that auxiliary, unlabelled data can be used in tandem with this approach to slow the decay in performance of these binary classifiers over time.

Most of existing manifold learning methods rely on Mean Squared Error (MSE) or \(\ell _2\) norm. However, for the problem of image quality assessment, these are not promising measure. In this paper, we introduce the concept of an image structure manifold which captures image structure features and discriminates image distortions. We propose a new manifold learning method, Locally Linear Image Structural Embedding (LLISE), and kernel LLISE for learning this manifold. The LLISE is inspired by Locally Linear Embedding (LLE) but uses SSIM rather than MSE. This paper builds a bridge between manifold learning and image fidelity assessment and it can open a new area for future investigations.

Hand gesture recognition have become versatile in numerous applications. In particular, the automotive industry has benefited from their deployment, and human-machine interface designers are using them to improve driver safety and comfort. In this paper, we investigate expanding the product segment of one of America’s top three automakers through deep learning to provide an increased driver convenience and comfort with the application of dynamic hand gesture recognition for vehicle self parking. We adapt the architecture of the end-to-end solution to expand the state of the art video classifier from a single image as input (fed by monocular camera) to a multiview 360 feed, offered by a six cameras module. Finally, we optimize the proposed solution to work on a limited resource embedded platform that is used by automakers for vehicle-based features, without sacrificing the accuracy robustness and real time functionality of the system.

Literature has shown that Mean Squared Error is not a promising measure for image fidelity and similarity assessment, and Structural Similarity Index (SSIM) can properly handle this aspect. The existing subspace learning methods in machine learning are based on Euclidean distance or MSE and thus cannot properly capture the structural features of images. In this paper, we define image structure subspace which captures the intrinsic structural features of an image and discriminates the different types of image distortions. Therefore, this paper provides a bridge between image processing and manifold learning opening future research opportunities. In order to learn this subspace, we propose SSIM kernel as a new kernel which can be used in kernel-based machine learning methods.

This paper provides initial results on developing a deep neural network-based system for driver distraction detection which is operational at daytime as well as nightime. Unlike other existing methods that rely on only RGB images for daytime detection, the proposed system consists of two operating modes. The daytime mode uses a convolutional neural network to classify drivers’ states based on their body poses in RGB images. The nighttime mode classifies Near Infrared images using a different neural network-based model and trained under different circumstances. To the best of our knowledge, this is the first work that explicitly addresses driver behavior detection at night using end-to-end convolutional neural networks. With initial experimental results, we empirically demonstrate that, with a relatively modest model complexity, the proposed system achieves high performance on driver distraction detection for both modes. Furthermore, we discuss the feasibility of developing a system with a small footprint and design structure but accurate enough to be deployed on a memory-restricted computing platform environment.

Despite the advances of deep learning in specific tasks using images, the principled assessment of image fidelity and similarity is still a critical ability to develop. As it has been shown that Mean Squared Error (MSE) is insufficient for this task, other measures have been developed with one of the most effective being Structural Similarity Index (SSIM). Such measures can be used for subspace learning but existing methods in machine learning, such as Principal Component Analysis (PCA), are based on Euclidean distance or MSE and thus cannot properly capture the structural features of images. In this paper, we define an image structure subspace which discriminates different types of image distortions. We propose Image Structural Component Analysis (ISCA) and also kernel ISCA by using SSIM, rather than Euclidean distance, in the formulation of PCA. This paper provides a bridge between image quality assessment and manifold learning opening a broad new area for future research.

This paper proposes an online intelligent demand coordination of plug-in electric vehicles (PEVs) in distribution systems. The proposed method is based on the assignment of scores to PEVs through a fuzzy expert system. As well, without violation of grid operational constraints, the PEVs are optimally charged in order to maximize the owners' satisfaction in terms of the energy delivered. The optimization problem of online PEV charging is defined as mixed-integer nonlinear programming. Simulation on a typical distribution network proves the effectiveness of the proposed methodology. Results of the analysis indicate that for more critical PEVs, which have shorter parking duration and higher required charging time, the proposed solution outperforms in more robust energy delivery to the PEV and, accordingly, more satisfaction for the owner.

Driver inattentiveness constitutes the main cause of road accidents, which makes it a major factor in road safety. In this paper, we propose a comprehensive framework to address the road safety problem by tackling it from a high-level information fusion standpoint, considering vehicular ad hoc networks (VANETs) as the deployment platform. The proposed framework relies on the multientity Bayesian networks (MEBNs), which exploit the expressiveness of first-order logic for semantic relations, and the strength of the Bayesian networks in handling uncertainty. First, the entities that influence the inattention phenomenon, as well as both their causal and semantic relationships, are identified. Next, an MEBN-based high-level information fusion framework is proposed through which entities, situations, and their relationships in specific contexts are modeled using MEBN fragments. Furthermore, MEBN inference is used to assess the situations of interest by estimating their states. To demonstrate the capabilities of the proposed framework, a collision warning system simulator has been developed, which evaluates the likelihood of a vehicle being in a near-collision situation using a wide variety of local and global information sources available in various VANET environments. If the threat of being in a near-collision situation is determined to be high, then the driver is warned accordingly. Our experimental results for two distinct single-vehicle and multivehicle categories of driving scenarios, as well as a novel hybrid MEBN inference, demonstrate the capability of the proposed framework to efficiently achieve situation and threat assessment on the road.

We have conducted a literature survey on situation awareness in connected cars.Situation awareness process is categorized into four main groups.Various taxonomies are proposed for different situation awareness technical tools.The capabilities of different situation awareness tools are compared.Major situation awareness frameworks in connected cars are introduced. Driving safety is among the most important factors in the design of next generation vehicles as an integral component of Intelligent Transportation Systems. Crash avoidance and reduction of potential subsequent fatalities require timely delivery of sensitive and pertinent safety information for the drivers. Hence, the driver can become aware of the current driving situation, and can consequently, take appropriate decisions to avoid potentially imminent hazards. In this paper, we propose a comprehensive survey on situation awareness within the context of connected vehicles and Internet of Cars (also called here as connected cars). We provide context for the Internet of Cars and highlight its major features. Furthermore, situation awareness in the Internet of Cars is explored through presenting an in-depth discussion on its different components. Various aspects of high and low level information fusions are described within this context. Besides, major methods/models in situation awareness are linked to the main aspects of each component, and an overall comparison between them is reported. Moreover, on-the-road safety frameworks incorporating situation awareness are highlighted. Finally, the challenging issues and the emerging trends that shall be faced by the research community are addressed.

This work addresses non-stationary traffic flow prediction by implementing an intelligent update scheme to deep neural networks. The intelligent update scheme works by monitoring the frequency domain features extracted from the traffic flow time series. The features at present are compared with the previous ones through a distance function. The resulting similarity is then fed to the exponentially weighted moving average to detect whether a notable change in the traffic flow is present or not. It has been shown in the experiments that the proposed method is able to handle the non-stationarity and produce acceptable traffic flow prediction. Moreover, the proposed method performance is comparable to the fully stochastic gradient descent update scheme while saving computational and time resources up to around 13%.

We propose a visual-based end to end lane following system which fuses temporal and spatial visual information to predict current and future control variables. Previous works only predict control variables for the next time point with the current visual information. In contrast, based on a long-term recurrent convolutional neural network, we investigate the effect of fusing history information of different lengths to predict the imminent control variable in different future horizons. Experimental results show that with long history visual information, the neural network can approximate human driving behaviours with high precision. Consistent with intuition is that the influence of history information declines as time moves forward. Meanwhile, history information of the past 0.6 seconds is of most information for the prediction, and the Mean Square Error (MSE) for the steering command prediction with 0.6s history information is $8.378 \times 10^{-3}\ m^{-1}$ . By training the model with control signals that lag behind visual information as targets, the testing result shows that it is possible to predict future control variables with great accuracy, while the best prediction accuracy happens to the steering command 0.4 seconds later.

Distracted driving is the main cause for car accidents. Driver inattention monitoring systems are promising solutions to mitigate this problem. In this paper, we propose a novel driver inattention monitoring system utilizing deep learning and fuzzy logic theory. A driver head pose estimation module is able to determine whether the driver is focusing on his frontal view, while a deep-learning-based distraction recognition module would detect whether the driver is performing a distraction activity. A danger level inference module based on fuzzy logic combines information from the head pose estimation and the distraction recognition modules to infer the danger level in a real-time manner. In the experimental work, a Convolutional Neural Network model is trained on data of high diversity allowing a more robust driver distraction detection compared to the model trained with only data collected by simulation experiments. In addition, we show that the proposed danger level inference strategy is an effective solution to detect dangerous driving situations by providing timely alerts depending on the vehicle speed.