Search

In this paper, we present tensor-based linear and nonlinear models for hyperspectral data classification and analysis. By exploiting the principles of tensor algebra, we introduce new classification architectures, the weight parameters of which satisfy the rank-1 canonical decomposition property. Then, we propose learning algorithms to train both linear and nonlinear classifiers. The advantages of the proposed classification approach are that: 1) it significantly reduces the number of weight parameters required to train the model (and thus the respective number of training samples); 2) it provides a physical interpretation of model coefficients on the classification output; and 3) it retains the spatial and spectral coherency of the input samples. The linear tensor-based model exploits the principles of logistic regression, assuming the rank-1 canonical decomposition property among its weights. For the nonlinear classifier, we propose a modification of a feedforward neural network (FNN), called rank-1 FNN, since its weights satisfy again the rank-1 canonical decomposition property. An appropriate learning algorithm is also proposed to train the network. Experimental results and comparisons with state-of-the-art classification methods, either linear (e.g., linear support vector machine) or nonlinear (e.g., deep learning), indicate the outperformance of the proposed scheme, especially in the cases where a small number of training samples is available. [ABSTRACT FROM AUTHOR]

We present a key frames selection algorithm based on three Iso-content principles (Iso-Content Distance, Iso-Content Error and Iso-Content Distortion), so that the selected key frames are equidistant-in video content according to the used principle. Two automatic approaches for defining the most appropriate number of key frames are proposed by exploiting supervised and unsupervised content criteria. Experimental results and the comparisons with existing methods from literature on large dataset of real-life video sequences illustrate the high performance of the proposed schemata. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new algorithm for fair scheduling, and we compare it to other scheduling schemes such as the Earliest Deadline First (EDF) and the First Come First Served (FCFS) schemes. Our algorithm uses a max-min fair sharing approach for providing fair access to users. When there is no shortage of resources, the algorithm assigns to each task enough computational power for it to finish within its deadline. When there is congestion, the main idea is to fairly reduce the CPU rates assigned to the tasks so that the share of resources that each user gets is proportional to the user's weight. The weight of a user may be defined as the user's contribution to the infrastructure or the price he is willing to pay for services or any other socioeconomic consideration. In our algorithms, all tasks whose requirements are lower than their fair share CPU rate are served at their demanded CPU rates. However, the CPU rates of tasks whose requirements are larger than their fair share CPU rate are reduced to fit the total available computational capacity in a fair manner. Three different versions of fair scheduling are adopted in this paper: the Simple Fair Task Order (SFTO), which schedules the tasks according to their respective fair completion times, the Adjusted Fair Task Order (AFTO), which refines the SFTO policy by ordering the tasks using the adjusted fair completion time, and the Max-Min Fair Share (MMFS) scheduling policy, which simultaneously addresses the problem of finding a fair task order and assigning a processor to each task based on a max-min fair sharing policy. Experimental results and comparisons with traditional scheduling schemes such as the EDF and the FCFS are presented using three different error criteria. Validation of the simulations using real experiments of tasks generated from 3D image- rendering processes is also provided. The three proposed scheduling schemes can be integrated into existing Grid computing architectures. [ABSTRACT FROM AUTHOR]

Abstract: Multimedia content modeling, i.e., identification of semantically meaningful entities, is an arduous task mainly due to the fact that (a) humans perceive the content using high-level concepts and (b) the subjectivity of human perception, which often interprets the same content in a different way at different times. For this reason, an efficient content management system has to be adapted to current user''s information needs and preferences through an on-line learning strategy based on users’ interaction. One adaptive learning strategy is relevance feedback, originally developed in traditional text-based information retrieval systems. In this way, the user interacts with the system to provide information about the relevance of the content, which is then fed back to the system to update its performance. In this paper, we evaluate and investigate three main types of relevance feedback algorithms; the Euclidean, the query point movements and the correlation-based approaches. In the first case, we examine heuristic and optimal techniques which are based either on the weighted or the generalized Euclidean distance. In the second case, we survey single and multipoint query movement schemes. As far as the third type is concerned, two different ways for parametrizing the normalized cross-correlation similarity metric are proposed. The first scales only the elements of the query feature vector and called query-scaling strategy, while the second scales both the query and the selected samples (query-sample scaling strategy). All the examined algorithms are evaluated using both subjective and objective criteria. Subjective evaluation is performed by depicting the best retrieved images as response of the system to a user''s query. Instead, objective evaluation is obtained using standard criteria, such as the precision–recall curve and the average normalized modified retrieval rank (ANMRR). Furthermore, a newly objective criterion, called average normalized similarity metric distance is introduced which exploits the difference among the actual and ideal similarity measure among all best retrievals. Discussions and comparisons of all the aforementioned relevance feedback algorithms are presented. [Copyright &y& Elsevier]

Abstract: In this paper, a multiscale content-based video decomposition scheme is presented for efficient non-linear (non-sequential) organization of the video visual content. In particular, each video file is analyzed in a multiscale structure of different “content resolution levels”, creating a hierarchy from the lowest (coarse) to the highest (fine) resolution. The scheme resembles the progressive transmission of still images, where instead of transmitting the image sequentially at a full resolution, by scanning it line by line, a lower image resolution is first delivered and then, the image quality gradually enhances so that the user is able at any time to see a preview of the image content. The proposed video decomposition is represented as a graph structure, each level of which corresponds to a particular content resolution, while the graph-nodes the respective regions that the content is analyzed at this level. Transitions among nodes of the same level are also permitted. The number of nodes at a given level expresses the degree of detail that the content at this level is analyzed. This number is estimated by minimizing the average transmitted information, required for localizing a video segment of interest and also takes into account the content complexity. Quality criteria are introduced to evaluate the efficiency of the proposed scheme. The efficiency of the organization is maximized if multiscale content decomposition is performed using content representatives and constructing content classes. Content representatives are estimated in our approach as the ones of the maximum dissimilarity, expressed by a distance metric. The optimization is conducted by incorporating a stochastic algorithm of logarithmically reduced searching area (stochastic logarithmic). Experimental results on real-life video sequences show that the proposed multiscale video organization enables users to detect content of interest much faster, compared to the conventional sequential video scanning or other video decomposition/summarization methods, resulting in a better organization efficiency as measured by the quality criteria. [Copyright &y& Elsevier]

In this paper, an interactive framework for navigating video sequences is presented using an optimal content-based video decomposition scheme. In particular, each video sequence is analyzed at different content resolution levels, creating a hierarchy from the lowest (coarse) to the highest (fine) resolution. This content hierarchy is represented as a tree structure, each level of which corresponds to a particular content resolution, while the tree nodes indicate the temporal video segments that the sequence content is partitioned at a given resolution. A criterion is introduced to measure the efficiency of the proposed scheme in organizing the video visual content and to compare it with other hierarchical video content representations and navigation schemes. The efficiency is measured as the difficulty for a user to locate a video segment of interest, while moving through different levels of hierarchy. In our case, video is decomposed so that the best efficiency is accomplished. However, the efficiency of a nonlinear video decomposition scheme depends on: 1) the number of paths required for a user to locate a relevant video segment and 2) the number of shot/frame classes (i.e., content representatives) extracted to represent the visual content. Both issues are addressed in this paper. In the first case, the probability of selecting a relevant video segment in the first path is maximized by extracting optimal content representatives through a minimization of a cross-correlation criterion. For the minimization, a genetic algorithm (GA) is adopted, since application of an exhaustive search to obtain the minimum value is too large to be implemented. The cross-correlation criterion is evaluated on the feature domain by extracting appropriate global and object-based descriptors for each video frame so that a better representation of the visual content is achieved. The second aspect (e.g., the number of content representatives) is addressed by minimizing the average transmitted information and simultaneously taking into consideration the temporal video segment complexity. More content representatives are extracted for video segments of high complexity, whereas a low number is required for low-complexity segments. In addition, a degree of interest is assigned to each video shot (or frame) to address the fact that, from the user's perception, the visual content of a set of shots (frames) satisfies his/her information needs. Finally, a computationally efficient algorithm is proposed to regulate the degree of detail (i.e., the number of shot/frames representatives) in case the visual content is not efficiently represented from the user's perceptive view. Experimental results on real-life video sequences indicate the performance of the proposed GA-based video decomposition scheme compared to other hierarchical video organization methods. [ABSTRACT FROM AUTHOR]

In this paper, a novel relevance feedback algorithm is proposed for improving the performance of inter active content-based retrieval systems. The algorithm recursively estimates the similarity measure, which is used for data ranking in description environments where similarity-based queries are applied, using a set of relevant/irrelevant samples feedback by the user to the system so that the adjusted response is a better approximation of the current user's information needs and preferences. In particulars using concepts of functional analysis, the similarity measure is expressed as a parametric form of known monotone increasing functional components. Then, the contribution of each functional component to the similarity measure is estimated through a recursive and efficient on-line learning algorithm so that: 1) the current user's needs and preferences, as indicated by a set of selected relevant/Irrelevant samples, are satisfied as much as possible, while simultaneously 2) a minimal modification of the already estimated similarity measure is accomplished. Experimental Salts on a large real-life database using objective evaluation criteria, such as the precision-recall curve and the average normalized modified retrieval rank (ANMRR), indicate that the proposed scheme outperforms the compared ones. In addition, the proposed algorithm requires low computational complexity and it can be implemented in a recursive way. [ABSTRACT FROM AUTHOR]

Proposes an unsupervised video object segmentation and tracking algorithm based on an adaptable neural-network architecture. Neural-network retraining strategy; Unsupervised retraining set estimation in videoconference sequences; Specifications of the decision mechanism; Experimental results; Conclusion.

The performance of a Content-Based Image Retrieval System (CBIR) depends on (a) the system's adaptability to the user's information needs, which permits different types of indexing and simultaneously reduces the subjectivity of human perception for the interpretation of the image visual content and (b) the efficient organization of the extracted descriptors, which represent the rich visual information. Both issues are addressed in this paper. Descriptor organization is performed using a fuzzy classification scheme fragmented into multidimensional classes, instead of the previous works where fuzzy histograms were created in one dimension using, for example, the feature vector norm. Multidimensionality relates the descriptors with one another and thus allows a compact and meaningful visual representation by mapping the elements of the resulted feature vectors with a physical visual interpretation. Furthermore, fuzzy classification is applied for all visual content descriptors, in contrast to the previous approaches where only color information is exploited. Two kinds of content descriptors are extracted in our case; global-based and region-based. The first refers to the global image characteristics, while the second exploits the region-based properties. Regions are obtained by applying a multiresolution implementation of the Recursive Shortest Spanning Tree (RSST) algorithm, called M-RSST in this paper. The second issue is addressed by proposing a computationally efficient relevance feedback mechanism based on an optimal weight updating strategy. The scheme relies on the cross-correlation measure, instead of the Euclidean distance which is mainly used in most relevance feedback algorithms. Crosscorrelation is a normalized measure, which expresses how similar the two feature vectors are and thus it indicates a metric of their content similarity. The proposed scheme can be recursively implemented in the case of multiple feedback iterations, instead of the previous... [ABSTRACT FROM AUTHOR]

Presents information on a study which described a neural network architecture for recursive nonlinear traffic prediction and modeling of MPEG video sources. Investigation of the problem of online traffic modeling; Implementation of the neural network architecture; Results and discussion; Conclusions.

In this paper, a combined neural network and an evolutionary programming scheme is proposed to improve the quality of wound core distribution transformers in an industrial environment by exploiting information derived from both the construction and transformer design phase. In particular, the neural network architecture is responsible for predicting transformer iron losses prior to their assembly, based on several actual core measurements, transformer design parameters and the specific core assembling. A genetic algorithm is applied to estimate the optimal core arrangement, (i.e. the way of core assembling) that yields a set of three-phase transformers of minimal iron losses. The minimization is performed by exploiting information derived from the neural network model resulting in a synergetic neural network-genetic algorithm scheme. After the transformer construction, the prediction accuracy of the neural network model is evaluated. If accuracy is poor, a weight adaptation algorithm is applied to improve the prediction performance. For the weight updating, both the current and the previous network knowledge are taken into account. Application of the proposed neural network-genetic algorithm scheme to our industrial environment indicates a significant reduction in the variation between the actual and the designed transformer iron losses. This further leads to a reduction of the production cost since a smaller safety margin can be used for the transformer design. [ABSTRACT FROM AUTHOR]

Deals with a study which proposed an algorithm for non-sequential video content representation. Depiction of the feature-based video representation; Description of the fuzzy classification of feature vectors; Presentation of experimental results using real life video sequences; Concluding remarks.

This paper presents an efficient technique for unsupervised content-based segmentation in stereoscopic video sequences by appropriately combined different content descriptors in a hierarchical framework. Three main modules are involved in the proposed scheme; extraction of reliable depth information, image partition into color and depth regions and a constrained fusion algorithm of color segments using information derived from the depth map. In the first module, each stereo pair is analyzed and the disparity field and depth map are estimated. Occlusion detection and compensation are also applied for improving the depth map estimation. In the following phase, color and depth regions are created using a novel complexity-reducing multiresolution implementation of the Recursive Shortest Spanning Tree algorithm (M-RSST). While depth segments provide a coarse representation of the image content, color regions describe very accurately object boundaries. For this reason, in the final phase, a new segmentation fusion algorithm is employed which projects color segments onto depth segments. Experimental results are presented which exhibit the efficiency of the proposed scheme as content-based descriptor, even in case of images with complicated visual content. [ABSTRACT FROM AUTHOR]

Features an efficient technique for summarization of stereoscopic video sequences which extracts a small but meaningful set of video frames using a content-based sampling algorithm. Capability of browsing digital stereoscopic video sequences; Efficient content-based queries and indexing; Recursive Shortest Spanning Tree algorithm; Multidimensional fuzzy classification of segment.

Proposes the layer modeling of Motion Picture Experts Group (MPEG)-1 coded video streams and statistical analysis of their characteristics at each layer. Basic characteristics of MPEG-1 encoders; Encoding algorithm; Study of autocorrelation function; Traffic model of frame layer; Intermediate layer modeling; Buffer configuration scheme; Prediction of video activity.

Presents information on a study which described an approach for improving the performance of neural-network classifiers in image recognition, segmentation or coding applications based on a retraining algorithm. Formulation of the problem; Optimal selection of the network retraining data; Decision mechanism for network retraining; Conclusions.

In this paper, we propose a Stack Auto-encoder (SAE)-Driven and Semi-Supervised (SSL)-Based Deep Neural Network (DNN) to extract buildings from relatively low-cost satellite near infrared images. The novelty of our scheme is that we employ only an extremely small portion of labeled data for training the deep model which constitutes less than 0.08% of the total data. This way, we significantly reduce the manual effort needed to complete an annotation process, and thus the time required for creating a reliable labeled dataset. On the contrary, we apply novel semi-supervised techniques to estimate soft labels (targets) of the vast amount of existing unlabeled data and then we utilize these soft estimates to improve model training. Overall, four SSL schemes are employed, the Anchor Graph, the Safe Semi-Supervised Regression (SAFER), the Squared-loss Mutual Information Regularization (SMIR), and an equal importance Weighted Average of them (WeiAve). To retain only the most meaning information of the input data, labeled and unlabeled ones, we also employ a Stack Autoencoder (SAE) trained under an unsupervised manner. This way, we handle noise in the input signals, attributed to dimensionality redundancy, without sacrificing meaningful information. Experimental results on the benchmarked dataset of Vaihingen city in Germany indicate that our approach outperforms all state-of-the-art methods in the field using the same type of color orthoimages, though the fact that a limited dataset is utilized (10 times less data or better, compared to other approaches), while our performance is close to the one achieved by high expensive and much more precise input information like the one derived from Light Detection and Ranging (LiDAR) sensors. In addition, the proposed approach can be easily expanded to handle any number of classes, including buildings, vegetation, and ground. [ABSTRACT FROM AUTHOR]

Presents an adaptive algorithm for extracting foreground objects from background in videophone or videoconference applications. Neural network architecture; Incorporation of the algorithm in motion-compensated discrete cosine transform-based coding schemes; Simulation results.

Non-Intrusive Load Monitoring (NILM) describes the process of inferring the consumption pattern of appliances by only having access to the aggregated household signal. Sequence-to-sequence deep learning models have been firmly established as state-of-the-art approaches for NILM, in an attempt to identify the pattern of the appliance power consumption signal into the aggregated power signal. Exceeding the limitations of recurrent models that have been widely used in sequential modeling, this paper proposes a transformer-based architecture for NILM. Our approach, called ELECTRIcity, utilizes transformer layers to accurately estimate the power signal of domestic appliances by relying entirely on attention mechanisms to extract global dependencies between the aggregate and the domestic appliance signals. Another additive value of the proposed model is that ELECTRIcity works with minimal dataset pre-processing and without requiring data balancing. Furthermore, ELECTRIcity introduces an efficient training routine compared to other traditional transformer-based architectures. According to this routine, ELECTRIcity splits model training into unsupervised pre-training and downstream task fine-tuning, which yields performance increases in both predictive accuracy and training time decrease. Experimental results indicate ELECTRIcity's superiority compared to several state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Regression algorithms are regularly used for improving the accuracy of satellite precipitation products. In this context, satellite precipitation and topography data are the predictor variables, and gauged-measured precipitation data are the dependent variables. Alongside this, it is increasingly recognised in many fields that combinations of algorithms through ensemble learning can lead to substantial predictive performance improvements. Still, a sufficient number of ensemble learners for improving the accuracy of satellite precipitation products and their large-scale comparison are currently missing from the literature. In this study, we work towards filling in this specific gap by proposing 11 new ensemble learners in the field and by extensively comparing them. We apply the ensemble learners to monthly data from the PERSIANN (Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks) and IMERG (Integrated Multi-satellitE Retrievals for GPM) gridded datasets that span over a 15-year period and over the entire contiguous United States (CONUS). We also use gauge-measured precipitation data from the Global Historical Climatology Network monthly database, version 2 (GHCNm). The ensemble learners combine the predictions of six machine learning regression algorithms (base learners), namely the multivariate adaptive regression splines (MARS), multivariate adaptive polynomial splines (poly-MARS), random forests (RF), gradient boosting machines (GBM), extreme gradient boosting (XGBoost) and Bayesian regularized neural networks (BRNN), and each of them is based on a different combiner. The combiners include the equal-weight combiner, the median combiner, two best learners and seven variants of a sophisticated stacking method. The latter stacks a regression algorithm on top of the base learners to combine their independent predictions. Its seven variants are defined by seven different regression algorithms, specifically the linear regression (LR) algorithm and the six algorithms also used as base learners. The results suggest that sophisticated stacking performs significantly better than the base learners, especially when applied using the LR algorithm. It also beats the simpler combination methods. [ABSTRACT FROM AUTHOR]

Detection of moving objects in videos is a crucial step toward successful surveillance and monitoring applications. A key component for such tasks is called background subtraction and tries to extract regions of interest from the image background for further processing or action. For this reason, its accuracy and real-time performance are of great significance. Although effective background subtraction methods have been proposed, only a few of them take into consideration the special characteristics of thermal imagery. In this paper, we propose a background subtraction scheme, which models the thermal responses of each pixel as a mixture of Gaussians with unknown number of components. Following a Bayesian approach, our method automatically estimates the mixture structure, while simultaneously it avoids over-/underfitting. The pixel density estimate is followed by an efficient and highly accurate updating mechanism, which permits our system to be automatically adapted to dynamically changing operation conditions. We propose a reference implementation of our method in reconfigurable hardware achieving both adequate performance and low-power consumption. Adopting a high-level synthesis design and demanding floating point arithmetic operations are mapped in reconfigurable hardware, demonstrating fast prototyping and on-field customization at the same time. [ABSTRACT FROM AUTHOR]

An important issue in performing dance analysis is the automatic extraction of its choreographic patterns, since these elements provide an abstract representation of the semantics of the dance and encode the overall dance storytelling. However, application of conventional video summarization algorithms on dance sequences cannot appropriately retrieve their choreographic patterns, since a dance is composed of an ordered set of sequential elements which are often repeated in time. Additionally, 3D geometry is lost using color information. For this reason, in this paper we propose a new dance summarization scheme of 3D motion captured data (in the form of skeleton joints coordinates) recorded using the Vicon motion capture system. The proposed key frame extraction method implements a hierarchical scheme that exploits spatio-temporal variations of dance features. Initially, global holistic descriptors are extracted to localize the key choreographic steps of a dance (coarse representation). Then, each segment is further decomposed into finer sub-segments to improve dance representativity (fine representation). The abstraction scheme exploits the concepts of a Sparse Modeling Representative Selection (SMRS) appropriately modified to enable spatio-temporal modelling of the dance sequences through a hierarchical decomposition algorithm. Our approach is evaluated on thirty folkloric dance sequences recorded at the Aristotle University of Thessaloniki under the framework of Terpsichore project representing five different choreographies and on publicly available datasets from Carnegie–Mellon University, which depict performances on theatrical kinesiology. Comparisons with other traditional video summarization methods indicate a clear superiority of the proposed hierarchical spatio-temporal decomposition scheme. [ABSTRACT FROM AUTHOR]

Over the last years deep learning methods have been shown to outperform previous state-of-the-art machine learning techniques in several fields, with computer vision being one of the most prominent cases. This review paper provides a brief overview of some of the most significant deep learning schemes used in computer vision problems, that is, Convolutional Neural Networks, Deep Boltzmann Machines and Deep Belief Networks, and Stacked Denoising Autoencoders. A brief account of their history, structure, advantages, and limitations is given, followed by a description of their applications in various computer vision tasks, such as object detection, face recognition, action and activity recognition, and human pose estimation. Finally, a brief overview is given of future directions in designing deep learning schemes for computer vision problems and the challenges involved therein. [ABSTRACT FROM AUTHOR]

Detection of outliers in radar signals is a considerable challenge in maritime surveillance applications. High-Frequency Surface-Wave (HFSW) radars have attracted significant interest as potential tools for long-range target identification and outlier detection at over-the-horizon (OTH) distances. However, a number of disadvantages, such as their low spatial resolution and presence of clutter, have a negative impact on their accuracy. In this paper, we explore the applicability of deep learning techniques for detecting deviations from the norm in behavioral patterns of vessels (outliers) as they are tracked from an OTH radar. The proposed methodology exploits the nonlinear mapping capabilities of deep stacked autoencoders in combination with density-based clustering. A comparative experimental evaluation of the approach shows promising results in terms of the proposed methodology’s performance. [ABSTRACT FROM AUTHOR]

In this study, we propose a tensor-based learning model to efficiently detect abnormalities on digital mammograms. Due to the fact that the availability of medical data is limited and often restricted by GDPR (general data protection regulation) compliance, the need for more sophisticated and less data-hungry approaches is urgent. Accordingly, our proposed artificial intelligence framework utilizes the canonical polyadic decomposition to decrease the trainable parameters of the wrapped Rank-R FNN model, leading to efficient learning using small amounts of data. Our model was evaluated on the open source digital mammographic database INBreast and compared with state-of-the-art models in this domain. The experimental results show that the proposed solution performs well in comparison with the other deep learning models, such as AlexNet and SqueezeNet, achieving 90% ± 4% accuracy and an F1 score of 84% ± 5%. Additionally, our framework tends to attain more robust performance with small numbers of data and is computationally lighter for inference purposes, due to the small number of trainable parameters. [ABSTRACT FROM AUTHOR]

Resource allocation and scheduling are fundamental issues in a Workflow Management System (WfMS). Effective resource management in WfMS should examine resource allocation together with task scheduling since these problems impose mutual constraints. Optimization of the one factor is subject to the other constraints and vice versa. Thus, an ideal algorithm should take into account not only performance metrics of the infrastructure, such as the number of resources and their utilization, but also quality criteria such as the percentage of tasks undergone violation in their temporal restrictions. In this paper, we propose an innovative algorithm which jointly optimizes the two aforementioned contradictory criteria. The algorithm, called Resource Conflicts Joint Optimization (Re.Co.Jo.Op.), minimizes resource conflicts subject to temporal constraints and simultaneously optimizes throughput or utilization subject to resources constraints. To achieve the optimization, the two factors are formulated in a matrix form and the optimal solution is found by applying concepts of the generalized eigenvalue analysis. A rough outline of an agent-based architecture is proposed to achieve runtime integration of our algorithm into a functional WfMS, while experimental results under different load environments and tasks assumption reveal the superiority of the proposed strategy than the other conventional approaches. [ABSTRACT FROM AUTHOR]

Non-intrusive load monitoring (NILM) is the task of disaggregating the total power consumption into its individual sub-components. Over the years, signal processing and machine learning algorithms have been combined to achieve this. Many publications and extensive research works are performed on energy disaggregation or NILM for the state-of-the-art methods to reach the desired performance. The initial interest of the scientific community to formulate and describe mathematically the NILM problem using machine learning tools has now shifted into a more practical NILM. Currently, we are in the mature NILM period where there is an attempt for NILM to be applied in real-life application scenarios. Thus, the complexity of the algorithms, transferability, reliability, practicality, and, in general, trustworthiness are the main issues of interest. This review narrows the gap between the early immature NILM era and the mature one. In particular, the paper provides a comprehensive literature review of the NILM methods for residential appliances only. The paper analyzes, summarizes, and presents the outcomes of a large number of recently published scholarly articles. Furthermore, the paper discusses the highlights of these methods and introduces the research dilemmas that should be taken into consideration by researchers to apply NILM methods. Finally, we show the need for transferring the traditional disaggregation models into a practical and trustworthy framework. [ABSTRACT FROM AUTHOR]

The COVID-19 pandemic has affected many aspects of human life around the world, due to its tremendous outcomes on public health and socio-economic activities. Policy makers have tried to develop efficient responses based on technologies and advanced pandemic control methodologies, to limit the wide spreading of the virus in urban areas. However, techniques such as social isolation and lockdown are short-term solutions that minimize the spread of the pandemic in cities and do not invert long-term issues that derive from climate change, air pollution and urban planning challenges that enhance the spreading ability. Thus, it seems crucial to understand what kind of factors assist or prevent the wide spreading of the virus. Although AI frameworks have a very efficient predictive ability as data-driven procedures, they often struggle to identify strong correlations among multidimensional data and provide robust explanations. In this paper, we propose the fusion of a heterogeneous, spatio-temporal dataset that combine data from eight European cities spanning from 1 January 2020 to 31 December 2021 and describe atmospheric, socio-economic, health, mobility and environmental factors all related to potential links with COVID-19. Remote sensing data are the key solution to monitor the availability on public green spaces between cities in the study period. So, we evaluate the benefits of NIR and RED bands of satellite images to calculate the NDVI and locate the percentage in vegetation cover on each city for each week of our 2-year study. This novel dataset is evaluated by a tree-based machine learning algorithm that utilizes ensemble learning and is trained to make robust predictions on daily cases and deaths. Comparisons with other machine learning techniques justify its robustness on the regression metrics RMSE and MAE. Furthermore, the explainable frameworks SHAP and LIME are utilized to locate potential positive or negative influence of the factors on global and local level, with respect to our model's predictive ability. A variation of SHAP, namely treeSHAP, is utilized for our tree-based algorithm to make fast and accurate explanations. [ABSTRACT FROM AUTHOR]

We present a S imultaneous P recise L ocalization A nd C lassification architecture employing a U-Net model (called SPLAC U-Net) for an automated simultaneous detection/localization of corrosion and recognition of rust grade from RGB images on metal structures. The proposed architecture is compatible with the innovative idea of automated maintenance and prefabrication since (i) rust detection is carried out at pixel level accuracy in a highly precise manner and (ii) localization/classification is performed simultaneously. Prior works on corrosion identification tend to focus either on localizing defected regions with bounding boxes, with low therefore localization precision, or on only visual inspection or classification and thus they are incompatible with the prefabrication concept. The proposed scheme consists of three layers. In the first layer, we utilize a lightweight deep U-Net model for semantically localizing corroded regions in images. In the second layer, we adopt a novel data projection scheme for refining the contours of the detected objects (i.e., defected regions) of the first layer, enhancing, therefore, localization precision. We find that the network after the boundary refinement process exceeds the typical deep learning methods performance on the accuracy, F1-score, and Intersection over Union (IoU) metrics. In the third and final layer of SPLAC U-Net, we extend the U-Net model by adding residual attention modules to efficiently classify multiple rust grades based on the ISO 8501-1 taxonomy. Experimental results using objective evaluation criteria, such as accuracy, precision, recall, F1-score, IoU, and Receiver Operating Characteristic Curve (ROC) show the outperformance of the proposed SLPAC U-Net in simultaneous detecting/localizing and classifying rust regions compared to other state-of-the-art methods exploiting either color distribution properties or traditional deep learning models. Lastly, a comparison of the computational performance shows that the total required time of the proposed SPLAC U-Net model (segmentation + projection refinement) is less than solely segmenting an image by using traditional deep learning models. • Corrosion localization and rust grade classification based on the ISO 8501-1 taxonomy, using a 3-layered U-Net architecture. • Projection-based boundary refinement to promote robotic-driven maintenance and prefabrication. • Comparison with color distribution methods shows an error reduction of 37% (F1-score) and 32% (IoU) in localizing corrosion. • Comparison with other deep learning methods shows an error reduction of 14% (F1-score) and 13% (IoU) in localizing corrosion. • Comparison with a conventional U-Net shows an error reduction of 28% (F1-score) and 22% (IoU) in classifying rust grades. [ABSTRACT FROM AUTHOR]

COVID-19 evolution imposes significant challenges for the European healthcare system. The heterogeneous spread of the pandemic within EU regions elicited a wide range of policies, such as school closure, transport restrictions, etc. However, the implementation of these interventions is not accompanied by the implementation of quantitative methods, which would indicate their effectiveness. As a result, the efficacy of such policies on reducing the spread of the virus varies significantly. This paper investigates the effectiveness of using deep learning paradigms to accurately model the spread of COVID-19. The deep learning approaches proposed in this paper are able to effectively map the temporal evolution of a COVID-19 outbreak, while simultaneously taking into account policy interventions directly into the modelling process. Thus, our approach facilitates data-driven decision making by utilizing previous knowledge to train models that predict not only the spread of COVID-19, but also the effect of specific policy measures on minimizing this spread. Global models at the EU level are proposed, which can be successfully applied at the national level. These models use various inputs in order to successfully model the spatio-temporal variability of the phenomenon and obtain generalization abilities. The proposed models are compared against the traditional epidemiological and Autoregressive Integrated Moving Average (ARIMA) models. [ABSTRACT FROM AUTHOR]

Modeling ionospheric variability throughout a proper total electron content (TEC) parameter estimation is a demanding, however, crucial, process for achieving better accuracy and rapid convergence in precise point positioning (PPP). In particular, the single-frequency PPP (SF-PPP) method lacks accuracy due to the difficulty of dealing adequately with the ionospheric error sources. In order to apply ionosphere corrections in techniques, such as SF-PPP, external information of global ionosphere maps (GIMs) is crucial. In this article, we propose a deep learning model to efficiently predict TEC values and to replace the GIM-derived data that inherently have a global character, with equal or better in accuracy regional ones. The proposed model is suitable for predicting the ionosphere delay at different locations of receiver stations. The model is tested during different periods of time, under different solar and geomagnetic conditions and for stations in various latitudes, providing robust estimations of the ionospheric activity at the regional level. Our proposed model is a hybrid model comprising of a 1-D convolutional layer used for the optimal feature extraction and stacked recurrent layers used for temporal time series modeling. Thus, the model achieves good performance in TEC modeling compared to other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Presents a method to reduce the iron losses of wound core distribution transformers based on a combined neural network-genetic algorithm approach. Iron loss reduction problem; Feasibility and practicality of the approach; Neural networks for predicting iron losses; Genetic algorithms for reducing iron losses.

This paper presents a technique for the detection of keratoconus via the construction of a 3D eye images from 2D frontal and lateral eye images. Keratoconus is a disease that affects the cornea. Normal case eyes have a round-shaped cornea, while patients who suffer from keratoconus have a cone-shaped cornea. Early diagnosis can decrease the risk of eyesight loss. Our aim is to create a method of fully automated keratoconus detection using digital-camera frontal and lateral eye images. The presented technique accurately determines case severity. Geometric features are extracted from 2D images to estimate depth information used to build 3D images of the cornea. The proposed methodology is easy to implement and time-efficient. 2D images of the eyes (frontal and lateral) are used as input, and 3D images from which the curvature of the cornea can be detected are produced as output. Our method involves two main steps: feature extraction and depth calculation. Machine learning from the 3D images dataset Dataverse, specifically taken by the Cornea/Anterior Segment OCT SS-1000 (CASIA), was performed. Results show that the method diagnosed the four stages of keratoconus (severe, moderate, mild, and normal) with an accuracy of 97.8%, as compared to manual diagnosis done by medical experts. [ABSTRACT FROM AUTHOR]

Recently, several educational game platforms have been proposed in the literature for choreographic training. However, their main limitation is that they fail to provide a quantitative assessment framework of a performing choreography against a groundtruth one. In this paper, we address this issue by proposing a machine learning framework exploiting deep learning paradigms. In particular, we introduce a long short-term memory network with the main capability of analyzing 3D captured skeleton feature joints of a dancer into predefined choreographic postures. This pose identification procedure is capable of providing a detailed (fine) evaluation score of a performing dance. In addition, the paper proposes a choreographic summarization architecture based on sparse modeling representative selection (SMRS) in order to abstractly represent the performing choreography through a set of key choreographic primitives. We have modified the SMRS algorithm in a way to extract hierarchies of key representatives. Choreographic summarization provides an efficient tool for a coarse quantitative evaluation of a dance. Moreover, hierarchical representation scheme allows for a scalable assessment of a choreography. The serious game platform supports advanced visualization toolkits using Labanotation in order to deliver the performing sequence in a formal documentation. [ABSTRACT FROM AUTHOR]

Serious games are receiving increasing attention in the field of cultural heritage (CH) applications. A special field of CH and education is intangible cultural heritage and particularly dance. Machine learning (ML) tools are necessary elements for the success of a serious game platform since they introduce intelligence in processing and analysis of users' interactivity. ML provides intelligent scoring and monitoring capabilities of the user's progress in a serious game platform. In this article, we introduce a deep learning model for motion primitive classification. The model combines a convolutional processing layer with a bidirectional analysis module. This way, RGB information is efficiently handled by the hierarchies of convolutions, while the bidirectional properties of a long short term memory (LSTM) model are retained. The resulting convolutionally enhanced bidirectional LSTM (CEBi-LSTM) architecture is less sensitive to skeleton errors, occurring using low-cost sensors, such as Kinect, while simultaneously handling the high amount of detail when using RGB visual information. [ABSTRACT FROM AUTHOR]

In this paper, we present WaterSpy, a project developing an innovative, compact, cost-effective photonic device for pervasive water quality sensing, operating in the mid-IR spectral range. The approach combines the use of advanced Quantum Cascade Lasers (QCLs) employing the Vernier effect, used as light source, with novel, fibre-coupled, fast and sensitive Higher Operation Temperature (HOT) photodetectors, used as sensors. These will be complemented by optimised laser driving and detector electronics, laser modulation and signal conditioning technologies. The paper presents the WaterSpy concept, the requirements elicited, the preliminary architecture design of the device, the use cases in which it will be validated, while highlighting the innovative technologies that contribute to the advancement of the current state of the art. [ABSTRACT FROM AUTHOR]

A system designed and developed for the three-dimensional (3-D) reconstruction of cultural heritage (CH) assets is presented. Two basic approaches are presented. The first one, resulting in an "approximate" 3-D model, uses images retrieved in online multimedia collections; it employs a clustering-based technique to perform content-based filtering and eliminate outliers that significantly reduce the performance of 3-D reconstruction frameworks. The second one is based on input image data acquired through terrestrial laser scanning, as well as close range and airborne photogrammetry; it follows a sophisticated multistep strategy, which leads to a "precise" 3-D model. Furthermore, the concept of change history maps is proposed to address the computational limitations involved in four-dimensional (4-D) modeling, i.e., capturing 3-D models of a CH landmark or site at different time instances. The system also comprises a presentation viewer, which manages the display of the multifaceted CH content collected and created. The described methods have been successfully applied and evaluated in challenging real-world scenarios, including the 4-D reconstruction of the historic Market Square of the German city of Calw in the context of the 4-D-CH-World EU project. [ABSTRACT FROM AUTHOR]

A genetically optimized neural detector is utilized for the identification of structural defects in concrete piles. The proposed methodology is applied on numerically generated data, involving two major defect types. A coupled finite element and scaled boundary finite element method approach is used to model the pile and its surrounding soil. The oscillation patterns, produced on the surface of the pile, depend strongly on the introduced defect type. The proposed defect detection system provides information about the type and the placement of the defect(s), given the surface’s oscillation patterns. [ABSTRACT FROM AUTHOR]

The Kalman filter represents a very popular signal processing tool, with a wide range of applications within many fields. Following a Bayesian framework, the Kalman filter recursively provides an optimal estimate of a set of unknown variables based on a set of noisy observations. Therefore, it fits system identification problems very well. Nevertheless, such scenarios become more challenging (in terms of the convergence and accuracy of the solution) when the parameter space becomes larger. In this context, the identification of linearly separable systems can be efficiently addressed by exploiting tensor-based decomposition techniques. Such multilinear forms can be modeled as rank-1 tensors, while the final solution is obtained by solving and combining low-dimension system identification problems related to the individual components of the tensor. Recently, the identification of multilinear forms was addressed based on the Wiener filter and most well-known adaptive algorithms. In this work, we propose a tensorial Kalman filter tailored to the identification of multilinear forms. Furthermore, we also show the connection between the proposed algorithm and other tensor-based adaptive filters. Simulation results support the theoretical findings and show the appealing performance features of the proposed Kalman filter for multilinear forms. [ABSTRACT FROM AUTHOR]

The degree of olive maturation is a very important factor to consider at harvest time, as it influences the organoleptic quality of the final product, for both oil and table use. The Jaén index, evaluated by measuring the average coloring of olive fruits (peel and pulp), is currently considered to be one of the most indicative methods to determine the olive ripening stage, but it is a slow assay and its results are not objective. The aim of this work is to identify the ripeness degree of olive lots through a real-time, repeatable, and objective machine vision method, which uses RGB image analysis based on a k-nearest neighbors classification algorithm. To overcome different lighting scenarios, pictures were subjected to an automatic colorimetric calibration method—an advanced 3D algorithm using known values. To check the performance of the automatic machine vision method, a comparison was made with two visual operator image evaluations. For 10 images, the number of black, green, and purple olives was also visually evaluated by these two operators. The accuracy of the method was 60%. The system could be easily implemented in a specific mobile app developed for the automatic assessment of olive ripeness directly in the field, for advanced georeferenced data analysis. [ABSTRACT FROM AUTHOR]

In recent years, the technology of artificial intelligence (AI) and robots is rapidly spreading to countries around the world. More and more scholars and industry experts have proposed AI deep learning models and methods to solve human life problems and improve work efficiency. Modern people's lives are very busy, which led us to investigate whether the demand for Bento buffet cafeterias has gradually increased in Taiwan. However, when eating at a buffet in a cafeteria, people often encounter two problems. The first problem is that customers need to queue up to check out after they have selected and filled their dishes from the buffet. However, it always takes too much time waiting, especially at lunch or dinner time. The second problem is sometimes customers question the charges calculated by cafeteria staff, claiming they are too expensive at the checkout counter. Therefore, it is necessary to develop an AI-enabled checkout system. The AI-enabled self-checkout system will help the Bento buffet cafeterias reduce long lineups without the need to add additional workers. In this paper, we used computer vision and deep-learning technology to design and implement an AI-enabled checkout system for Bento buffet cafeterias. The prototype contains an angle steel shelf, a Kinect camera, a light source, and a desktop computer. Six baseline convolutional neural networks were applied for comparison on food recognition. In our experiments, there were 22 different food categories in a Bento buffet cafeteria employed. Experimental results show that the inception_v4 model can achieve the highest average validation accuracy of 99.11% on food recognition, but it requires the most training and recognition time. AlexNet model achieves a 94.5% accuracy and requires the least training time and recognition time. We propose a hierarchical approach with two stages to achieve good performance in both the recognition accuracy rate and the required training and recognition time. The approach is designed to perform the first step of identification and the second step of recognizing similar food images, respectively. Experimental results show that the proposed approach can achieve a 96.3% accuracy rate on our test dataset and required very little recognition time for input images. In addition, food volumes could be estimated using the depth images captured by the Kinect camera, and a framework of visual checkout system was successfully built. [ABSTRACT FROM AUTHOR]

The pavement inspection task, which mainly includes crack and garbage detection, is essential and carried out frequently. The human-based or dedicated system approach for inspection can be easily carried out by integrating with the pavement sweeping machines. This work proposes a deep learning-based pavement inspection framework for self-reconfigurable robot named Panthera. Semantic segmentation framework SegNet was adopted to segment the pavement region from other objects. Deep Convolutional Neural Network (DCNN) based object detection is used to detect and localize pavement defects and garbage. Furthermore, Mobile Mapping System (MMS) was adopted for the geotagging of the defects. The proposed system was implemented and tested with the Panthera robot having NVIDIA GPU cards. The experimental results showed that the proposed technique identifies the pavement defects and litters or garbage detection with high accuracy. The experimental results on the crack and garbage detection are presented. It is found that the proposed technique is suitable for deployment in real-time for garbage detection and, eventually, sweeping or cleaning tasks. [ABSTRACT FROM AUTHOR]

Obtaining accurate global motion is a crucial step for video stabilization. This paper proposes a robust and simple method to implement global motion estimation. We don't extend the framework of 2D video stabilization but add a "plug and play" module to motion estimation based on feature points. Firstly, simple linear iterative clustering (SLIC) pre-segmentation is used to obtain superpixels of the video frame, clustering is performed according to the superpixel centroid motion vector and cluster center with large value is eliminated. Secondly, in order to obtain accurate global motion estimation, an improved K-means clustering is proposed. We match the feature points of the remaining superpixels between two adjacent frames, establish a feature points' motion vector space, and use improved K-means clustering for clustering. Finally, the richest cluster is being retained, and the global motion is obtained by homography transformation. Our proposed method has been verified on different types of videos and has efficient performance than traditional approaches. The stabilization video has an average improvement of 0.24 in the structural similarity index than the original video and 0.1 higher than the traditional method. [ABSTRACT FROM AUTHOR]

Recent studies indicate that detecting radiographic patterns on CT chest scans can yield high sensitivity and specificity for COVID-19 identification. In this paper, we scrutinize the effectiveness of deep learning models for semantic segmentation of pneumonia-infected area segmentation in CT images for the detection of COVID-19. Traditional methods for CT scan segmentation exploit a supervised learning paradigm, so they (a) require large volumes of data for their training, and (b) assume fixed (static) network weights once the training procedure has been completed. Recently, to overcome these difficulties, few-shot learning (FSL) has been introduced as a general concept of network model training using a very small amount of samples. In this paper, we explore the efficacy of few-shot learning in U-Net architectures, allowing for a dynamic fine-tuning of the network weights as new few samples are being fed into the U-Net. Experimental results indicate improvement in the segmentation accuracy of identifying COVID-19 infected regions. In particular, using 4-fold cross-validation results of the different classifiers, we observed an improvement of 5.388 ± 3.046% for all test data regarding the IoU metric and a similar increment of 5.394 ± 3.015% for the F1 score. Moreover, the statistical significance of the improvement obtained using our proposed few-shot U-Net architecture compared with the traditional U-Net model was confirmed by applying the Kruskal-Wallis test (p-value = 0.026). [ABSTRACT FROM AUTHOR]

The virtual inpainting of artworks provides a nondestructive mode of hypothesis visualization, and it is especially attractive when physical restoration raises too many methodological and ethical concerns. At the same time, in Cultural Heritage applications, the level of details in virtual reconstruction and their accuracy are crucial. We propose an inpainting algorithm that is based on generative adversarial network, with two generators: one for edges and another one for colors. The color generator rebalances chromatically the result by enforcing a loss in the discretized gamut space of the dataset. This way, our method follows the modus operandi of an artist: edges first, then color palette, and, at last, color tones. Moreover, we simulate the stochasticity of the lacunae in artworks with morphological variations of a random walk mask that recreate various degradations, including craquelure. We showcase the performance of our model on a dataset of digital images of wall paintings from the Dunhuang UNESCO heritage site. Our proposals of restored images are visually satisfactory and they are quantitatively comparable to state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

The astounding development of optical sensing imaging technology, coupled with the impressive improvements in machine learning algorithms, has increased our ability to understand and extract information from scenic events. In most cases, Convolution neural networks (CNNs) are largely adopted to infer knowledge due to their surprising success in automation, surveillance, and many other application domains. However, the convolution operations' overwhelming computation demand has somewhat limited their use in remote sensing edge devices. In these platforms, real-time processing remains a challenging task due to the tight constraints on resources and power. Here, the transfer and processing of non-relevant image pixels act as a bottleneck on the entire system. It is possible to overcome this bottleneck by exploiting the high bandwidth available at the sensor interface by designing a CNN inference architecture near the sensor. This paper presents an attention-based pixel processing architecture to facilitate the CNN inference near the image sensor. We propose an efficient computation method to reduce the dynamic power by decreasing the overall computation of the convolution operations. The proposed method reduces redundancies by using a hierarchical optimization approach. The approach minimizes power consumption for convolution operations by exploiting the Spatio-temporal redundancies found in the incoming feature maps and performs computations only on selected regions based on their relevance score. The proposed design addresses problems related to the mapping of computations onto an array of processing elements (PEs) and introduces a suitable network structure for communication. The PEs are highly optimized to provide low latency and power for CNN applications. While designing the model, we exploit the concepts of biological vision systems to reduce computation and energy. We prototype the model in a Virtex UltraScale+ FPGA and implement it in Application Specific Integrated Circuit (ASIC) using the TSMC 90nm technology library. The results suggest that the proposed architecture significantly reduces dynamic power consumption and achieves high-speed up surpassing existing embedded processors' computational capabilities. [ABSTRACT FROM AUTHOR]

Among aquatic biota, corals provide shelter with sufficient nutrition to a wide variety of underwater life. However, a severe decline in the coral resources can be noted in the last decades due to global environmental changes causing marine pollution. Hence, it is of paramount importance to develop and deploy swift coral monitoring system to alleviate the destruction of corals. Performing semantic segmentation on underwater images is one of the most efficient methods for automatic investigation of corals. Firstly, to design a coral investigation system, RGB and spectral images of various types of corals in natural and artificial aquatic sites are collected. Based on single-channel images, a convolutional neural network (CNN) model, named DeeperLabC, is employed for the semantic segmentation of corals, which is a concise and modified deeperlab model with encoder-decoder architecture. Using ResNet34 as a skeleton network, the proposed model extracts coral features in the images and performs semantic segmentation. DeeperLabC achieved state-of-the-art coral segmentation with an overall mean intersection over union (IoU) value of 93.90%, and maximum F1-score of 97.10% which surpassed other existing benchmark neural networks for semantic segmentation. The class activation map (CAM) module also proved the excellent performance of the DeeperLabC model in binary classification among coral and non-coral bodies. [ABSTRACT FROM AUTHOR]

Regular washing of public pavements is necessary to ensure that the public environment is sanitary for social activities. This is a challenge for autonomous cleaning robots, as they must adapt to the environment with varying pavement widths while avoiding pedestrians. A self-reconfigurable pavement sweeping robot, named Panthera, has the mechanisms to perform reconfiguration in width to enable smooth cleaning operations, and it changes its behavior based on environment dynamics of moving pedestrians and changing pavement widths. Reconfiguration in the robot's width is possible, due to the scissor mechanism at the core of the robot's body, which is driven by a lead screw motor. Panthera will perform locomotion and reconfiguration based on perception sensors feedback control proposed while using an Red Green Blue-D (RGB-D) camera. The proposed control scheme involves publishing robot kinematic parameters for reconfiguration during locomotion. Experiments were conducted in outdoor pavements to demonstrate the autonomous reconfiguration during locomotion to avoid pedestrians while complying with varying pavements widths in a real-world scenario. [ABSTRACT FROM AUTHOR]

This paper presents an algorithm for segmentation and shape analysis of erythrocyte images collected using an optical microscope. The main objective of the proposed approach is to compute statistical object values such as the number of erythrocytes in the image, their size, and width to height ratio. A median filter, a mean filter and a bilateral filter were used for initial noise reduction. Background subtraction using a rolling ball filter removes background irregularities. Combining the distance transform with the Otsu and watershed segmentation methods allows for initial image segmentation. Further processing steps, including morphological transforms and the previously mentioned segmentation methods, were applied to each segmented cell, resulting in an accurate segmentation. Finally, the noise standard deviation, sensitivity, specificity, precision, negative predictive value, accuracy and the number of detected objects are calculated. The presented approach shows that the second stage of the two-stage segmentation algorithm applied to individual cells segmented in the first stage allows increasing the precision from 0.857 to 0.968 for the artificial image example tested in this paper. The next step of the algorithm is to categorize segmented erythrocytes to identify poorly segmented and abnormal ones, thus automating this process, previously often done manually by specialists. The presented segmentation technique is also applicable as a probability map processor in the deep learning pipeline. The presented two-stage processing introduces a promising fusion model presented by the authors for the first time. [ABSTRACT FROM AUTHOR]