Search

Reservoir computing (RC) is a popular class of recurrent neural networks (RNNs) with untrained dynamics. Recently, advancements on deep RC architectures have shown a great impact in time-series applications, showing a convenient trade-off between predictive performance and required training complexity. In this paper, we go more in depth into the analysis of untrained RNNs by studying the quality of recurrent dynamics developed by the layers of deep RC neural networks. We do so by assessing the richness of the neural representations in the different levels of the architecture, using measures originating from the fields of dynamical systems, numerical analysis and information theory. Our experiments, on both synthetic and real-world datasets, show that depth—as an architectural factor of RNNs design—has a natural effect on the quality of RNN dynamics (even without learning of the internal connections). The interplay between depth and the values of RC scaling hyper-parameters, especially the scaling of inter-layer connections, is crucial to design rich untrained recurrent neural systems. [ABSTRACT FROM AUTHOR]

Motivation Dynamical properties of biochemical pathways (BPs) help in understanding the functioning of living cells. Their in silico assessment requires simulating a dynamical system with a large number of parameters such as kinetic constants and species concentrations. Such simulations are based on numerical methods that can be time-expensive for large BPs. Moreover, parameters are often unknown and need to be estimated. Results We developed a framework for the prediction of dynamical properties of BPs directly from the structure of their graph representation. We represent BPs as Petri nets, which can be automatically generated, for instance, from standard SBML representations. The core of the framework is a neural network for graphs that extracts relevant information directly from the Petri net structure and exploits them to learn the association with the desired dynamical property. We show experimentally that the proposed approach reliably predicts a range of diverse dynamical properties (robustness, monotonicity, and sensitivity) while being faster than numerical methods at prediction time. In synergy with the neural network models, we propose a methodology based on Petri nets arc knock-out that allows the role of each molecule in the occurrence of a certain dynamical property to be better elucidated. The methodology also provides insights useful for interpreting the predictions made by the model. The results support the conjecture often considered in the context of systems biology that the BP structure plays a primary role in the assessment of its dynamical properties. Availability and implementation https://github.com/marcopodda/petri-bio (code), https://zenodo.org/record/7610382 (data). [ABSTRACT FROM AUTHOR]

[Display omitted] • Introduced DynGESN, a deep reservoir computing model for discrete-time dynamic graphs • Proved a theoretical sufficient condition for the echo state property of DynGESN • Best efficiency/accuracy trade-off w.r.t. temporal graph kernels and neural networks Relations between entities evolving through discrete time-steps can be represented by discrete-time dynamic graphs. Examples include hourly interactions between social network users or daily infection spreading. In this paper, we present Dynamic Graph Echo State Network (DynGESN), a reservoir computing model for the efficient processing of discrete-time dynamic temporal graphs. We prove a sufficient condition for the echo state property, which ensures that graph embeddings are independent of initial conditions, and we briefly analyze reservoir dynamics. DynGESN is compared against temporal graph kernels (TGKs) on twelve graph classification tasks, and against ten different end-to-end trained temporal graph convolutional networks (TGNs) on four vertex regression tasks, since TGKs are limited to graph-level tasks. Compared to TGKs that need to hold the entire history of vertex interactions, our model provides a vector encoding for the dynamic graph that is updated at each time-step without requiring training. Experiments show that our model achieves accuracy in line with TGKs that have comparable computational complexity, while still offering space and time requirements better suited to scale to large-size data. Moreover, DynGESN overall achieves superior or on par accuracy with respect to TGNs, while improving efficiency by up ten times on inference and training time. [ABSTRACT FROM AUTHOR]

• We propose to generate graphs node by node in a sequential fashion. • Generation is carried out by two Recurrent Neural Networks that sample the nodes attached to an edge sequentially. • Our model is capable of producing novel and unique graphs that retain structural features of those in the training set. Graph generation with Machine Learning is an open problem with applications in various research fields. In this work, we propose to cast the generative process of a graph into a sequential one, relying on a node ordering procedure. We use this sequential process to design a novel generative model composed of two recurrent neural networks that learn to predict the edges of graphs: the first network generates one endpoint of each edge, while the second network generates the other endpoint conditioned on the state of the first. We test our approach extensively on five different datasets, comparing with two well-known baselines coming from graph literature, and two recurrent approaches, one of which holds state of the art performances. Evaluation is conducted considering quantitative and qualitative characteristics of the generated samples. Results show that our approach is able to yield novel, and unique graphs originating from very different distributions while retaining structural properties very similar to those in the training sample. Under the proposed evaluation framework, our approach is able to reach performances comparable to the current state of the art on the graph generation task. [ABSTRACT FROM AUTHOR]

[Display omitted] • Extended Graph Echo State Network (GESN) to supervised node classification tasks. • Improved efficiency/accuracy trade-off on medium- and large-scale heterophilic graphs. • Reservoirs outside stability constraints can be more effective for node-level tasks. • GESN allows effective node encoding by message-passing up to the largest shortest path. Node classification tasks on graphs are addressed via fully-trained deep message-passing models that learn a hierarchy of node representations via multiple aggregations of a node's neighbourhood. While effective on graphs that exhibit a high ratio of intra-class edges, this approach poses challenges in the opposite case, i.e. heterophily, where nodes belonging to the same class are usually further apart. In graphs with a high degree of heterophily, the smoothed representations based on close neighbours computed by convolutional models are no longer effective. So far, architectural variations in message-passing models to reduce excessive smoothing or rewiring the input graph to improve longer-range message passing have been proposed. In this paper, we address the challenges of heterophilic graphs with Graph Echo State Network (GESN) for node classification. GESN is a reservoir computing model for graphs, where node embeddings are recursively computed by an untrained message-passing function. Our experiments show that reservoir models are able to achieve better or comparable accuracy with respect to most fully trained deep models that implement ad hoc variations in the architectural bias or perform rewiring as a preprocessing step on the input graph, with an improvement in terms of efficiency/accuracy trade-off. Furthermore, our analysis shows that GESN is able to effectively encode the structural relationships of a graph node, by showing a correlation between iterations of the recursive embedding function and the distribution of shortest paths in a graph. [ABSTRACT FROM AUTHOR]

Abstract In this paper, we provide a novel approach to the architectural design of deep Recurrent Neural Networks using signal frequency analysis. In particular, focusing on the Reservoir Computing framework and inspired by the principles related to the inherent effect of layering, we address a fundamental open issue in deep learning, namely the question of how to establish the number of layers in recurrent architectures in the form of deep echo state networks (DeepESNs). The proposed method is first analyzed and refined on a controlled scenario and then it is experimentally assessed on challenging real-world tasks. The achieved results also show the ability of properly designed DeepESNs to outperform RC approaches on a speech recognition task, and to compete with the state-of-the-art in time-series prediction on polyphonic music tasks. Highlights • We address the issue of architectural design for Deep Recurrent Neural Networks. • We focus on the frequency analysis for layering design. • We propose an efficient approach to choose the number of recurrent layers in DeepESN. • We show the empirical advantage of very Deep (> 30 recurrent layers) RNNs. [ABSTRACT FROM AUTHOR]

The analysis of deep Recurrent Neural Network (RNN) models represents a research area of increasing interest. In this context, the recent introduction of Deep Echo State Networks (DeepESNs) within the Reservoir Computing paradigm, enabled to study the intrinsic properties of hierarchically organized RNN architectures.In this paper we investigate the DeepESN model under a dynamical system perspective, aiming at characterizing the important aspect of stability of layered recurrent dynamics excited by external input signals.To this purpose, we develop a framework based on the study of the local Lyapunov exponents of stacked recurrent models, enabling the analysis and control of the resulting dynamical regimes. The introduced framework is demonstrated on artificial as well as real-world datasets. The results of our analysis on DeepESNs provide interesting insights on the real effect of layering in RNNs. In particular, they show that when recurrent units are organized in layers, then the resulting network intrinsically develops a richer dynamical behavior that is naturally driven closer to the edge of criticality. As confirmed by experiments on the short-term Memory Capacity task, this characterization makes the layered design effective, with respect to the shallow counterpart with the same number of units, especially in tasks that require much in terms of memory. [ABSTRACT FROM AUTHOR]

In this paper, we propose an empirical analysis of deep recurrent neural network (RNN) architectures with stacked layers. The main aim is to address some fundamental open research issues on the significance of creating deep layered architectures in RNN and to characterize the inherent hierarchical representation of time in such models, especially for efficient implementations. In particular, the analysis aims at the study and proposal of approaches to develop and enhance hierarchical dynamics in deep architectures within the efficient Reservoir Computing (RC) framework for RNN modeling. The effect of a deep layered organization of RC models is investigated in terms of both occurrence of multiple time-scale and increasing of richness of the dynamics. It turns out that a deep layering of recurrent models allows an effective diversification of temporal representations in the layers of the hierarchy, by amplifying the effects of the factors influencing the time-scales and the richness of the dynamics, measured as the entropy of recurrent units activations. The advantages of the proposed approach are also highlighted by measuring the increment of the short-term memory capacity of the RC models. [ABSTRACT FROM AUTHOR]

Where, when and how much animals eat provide valuable insights into their ecology. In this paper, we present a comparative analysis between Support Vector Machine (SVM) and Input Delay Neural Network (IDNN) models to identify prey capture events from penguin accelerometry data. A pre-classified dataset of 3D time-series data from back-mounted accelerometers was used. We trained both the models to classify the penguins’ behavior at intervals as either ‘prey handling’ or ‘swimming’. The aim was to determine whether IDNN could achieve the same level of classification accuracy as SVM, but with reduced memory demands. This would enable the IDNN model to be embedded on the accelerometer micro-system itself, and hence reduce the magnitude of the output data to be uploaded. Based on the classification results, this paper provides an analysis of the two models from both an accuracy and applicability point of view. The experimental results show that both models achieve an equivalent accuracy of approx. 85% using the featured data, with a memory demand of 0.5 kB for IDNN and 0.7 Mb for SVM. The raw accelerometer data let us improve the generalizability of the models with a slightly lower accuracy to around 80%. This indicates that the IDNN model can embed on the accelerometer itself, reducing problems associated with raw time-series data retrieval and loss. [ABSTRACT FROM AUTHOR]

Abstract: The paper introduces an input-driven generative model for tree-structured data that extends the bottom-up hidden tree Markov model to non-homogeneous state transition and emission probabilities. We show how the proposed input-driven approach can be used to realize different types of structured transductions between trees. A thorough experimental analysis is proposed to investigate the advantage of introducing an input-driven dynamics in structured-data processing. The results of this analysis suggest that input-driven models can capture more discriminative structural information than homogeneous approaches in computational learning tasks, including document classification and more general substructure categorization. [Copyright &y& Elsevier]

Abstract: In this paper we present the Tree Echo State Network (TreeESN) model, generalizing the paradigm of Reservoir Computing to tree structured data. TreeESNs exploit an untrained generalized recursive reservoir, exhibiting extreme efficiency for learning in structured domains. In addition, we highlight through the paper other characteristics of the approach: First, we discuss the Markovian characterization of reservoir dynamics, extended to the case of tree domains, that is implied by the contractive setting of the TreeESN state transition function. Second, we study two types of state mapping functions to map the tree structured state of TreeESN into a fixed-size feature representation for classification or regression tasks. The critical role of the relation between the choice of the state mapping function and the Markovian characterization of the task is analyzed and experimentally investigated on both artificial and real-world tasks. Finally, experimental results on benchmark and real-world tasks show that the TreeESN approach, in spite of its efficiency, can achieve comparable results with state-of-the-art, although more complex, neural and kernel based models for tree structured data. [Copyright &y& Elsevier]

Abstract: Echo State Networks (ESNs) constitute an emerging approach for efficiently modeling Recurrent Neural Networks (RNNs). In this paper we investigate some of the main aspects that can be accounted for the success and limitations of this class of models. In particular, we propose complementary classes of factors related to contractivity and architecture of reservoirs and we study their relative relevance. First, we show the existence of a class of tasks for which ESN performance is independent of the architectural design. The effect of the Markovian factor, characterizing a significant class within these cases, is shown by introducing instances of easy/hard tasks for ESNs featured by contractivity of reservoir dynamics. In the complementary cases, for which architectural design is effective, we investigate and decompose the aspects of network design that allow a larger reservoir to progressively improve the predictive performance. In particular, we introduce four key architectural factors: input variability, multiple time-scales dynamics, non-linear interactions among units and regression in an augmented feature space. To investigate the quantitative effects of the different architectural factors within this class of tasks successfully approached by ESNs, variants of the basic ESN model are proposed and tested on instances of datasets of different nature and difficulty. Experimental evidences confirm the role of the Markovian factor and show that all the identified key architectural factors have a major role in determining ESN performances. [Copyright &y& Elsevier]

We propose a new method based on a Recursive Neural Network (RecNN) for predicting polymer properties from their structured molecular representations. RecNN allows for a completely novel approach to QSPR analysis by direct adaptive processing of molecular graphs. This model joins the representational power of structured domains with Neural Network ability to capture underlying complex relationships in the data by a process of training from examples. To this aim, a structured representation was designed for the modelling of polymer structures. The adopted representation can account also for average macromolecule characteristics, such as degree of polymerization, stereoregularity, comonomer distribution. To begin with, this model was applied to the prediction of the glass transition temperature of (meth)acrylic polymers with different degree of main chain tacticity. The results so far obtained indicate that the proposed representation of polymer structure can convey information on both the repeating unit structure and average polymer features. The ability of the proposed RecNN method of treating this structured representation makes this method more general and flexible with respect to standard literature methods. Moreover, the same model can handle at the same time the Tg of polymer samples present in only one tacticity form together with that of polymer with different stereoregularity. [ABSTRACT FROM AUTHOR]

This paper presents a new approach for learning in structured domains (SDs) using a constructive neural network for graphs (NN4G). The new model allows the extension of the input domain for supervised neural networks to a general class of graphs including both acyclic/cyclic, directed/undirected labeled graphs. In particular, the model can realize adaptive contextual transductions, learning the mapping from graphs for both classification and regression tasks. In contrast to previous neural networks for structures that had a recursive dynamics, NN4G is based on a constructive feedforward architecture with state variables that uses neurons with no feedback connections. The neurons are applied to the input graphs by a general traversal process that relaxes the constraints of previous approaches derived by the causality assumption over hierarchical input data. Moreover, the incremental approach eliminates the need to introduce cyclic dependencies in the definition of the system state variables. In the traversal process, the NN4G units exploit (local) contextual information of the graphs vertices. In spite of the simplicity of the approach, we show that, through the compositionality of the contextual information developed by the learning, the model can deal with contextual information that is incrementally extended according to the graphs topology. The effectiveness and the generality of the new approach are investigated by analyzing its theoretical properties and providing experimental results. [ABSTRACT FROM AUTHOR]

Summary: Objective: Modeling phylogenetic interactions is an open issue in many computational biology problems. In the context of gene function prediction we introduce a class of kernels for structured data leveraging on a hierarchical probabilistic modeling of phylogeny among species. Methods and materials: We derive three kernels belonging to this setting: a sufficient statistics kernel, a Fisher kernel, and a probability product kernel. The new kernels are used in the context of support vector machine learning. The kernels adaptivity is obtained through the estimation of the parameters of a tree structured model of evolution using as observed data phylogenetic profiles encoding the presence or absence of specific genes in a set of fully sequenced genomes. Results: We report results obtained in the prediction of the functional class of the proteins of the budding yeast Saccharomyces cerevisae which favorably compare to a standard vector based kernel and to a non-adaptive tree kernel function. A further comparative analysis is performed in order to assess the impact of the different components of the proposed approach. Conclusions: We show that the key features of the proposed kernels are the adaptivity to the input domain and the ability to deal with structured data interpreted through a graphical model representation. [Copyright &y& Elsevier]

Cascade correlation (CC) constitutes a training method for neural networks that determines the weights as well as the neural architecture during training. Various extensions of CC to structured data have been proposed: recurrent cascade correlation (RCC) for sequences, recursive cascade correlation (RecCC) for tree structures with limited fan-out, and contextual recursive cascade correlation (CRecCC) for rooted directed positional acyclic graphs (DPAGs) with limited fan-in and fan-out. We show that these models possess the universal approximation property in the following sense: given a probability measure P on the input set, every measurable function from sequences into a real vector space can be approximated by a sigmoidal RCC up to any desired degree of accuracy up to inputs of arbitrary small probability. Every measurable function from tree structures with limited fan-out into a real vector space can be approximated by a sigmoidal RecCC with multiplicative neurons up to any desired degree of accuracy up to inputs of arbitrary small probability. For sigmoidal CRecCC networks with multiplicative neurons, we show the universal approximation capability for functions on an important subset of all DPAGs with limited fan-in and fan-out for which a specific linear representation yields unique codes. We give one sufficient structural condition for the latter property, which can easily be tested: the enumeration of ingoing and outgoing edges should be compatible. This property can be fulfilled for every DPAG with fan-in and fan-out two via reenumeration of children and parents, and for larger fan-in and fan-out via an expansion of the fan-in and fan-out and reenumeration of children and parents. In addition, the result can be generalized to the case of input-output isomorphic transductions of structures. Thus, CRecCC networks constitute the first neural models for which the universal approximation capability of functions involving fairly general acyclic graph structures is proved. [ABSTRACT FROM AUTHOR]

Abstract: The aim of this paper is to start a comparison between recursive neural networks (RecNN) and kernel methods for structured data, specifically support vector regression (SVR) machine using a tree kernel, in the context of regression tasks for trees. Both the approaches can deal directly with a structured input representation and differ in the construction of the feature space from structured data. We present and discuss preliminary empirical results for specific regression tasks involving well-known quantitative structure-activity and quantitative structure-property relationship (QSAR/QSPR) problems, where both the approaches are able to achieve state-of-the-art results. [Copyright &y& Elsevier]

This paper propose a first approach to deal with contextual information in structured domains by recursive neural networks The proposed model, i.e., contextual recursive cascade correlation (CRC C), a generalization of the recursive cascade cor- relation (RCC) model, is able to partially remove the causality assumption by exploiting contextual information stored in frozen units. We formally characterize the properties of CRCC showing that it is able to compute contextual transductions and also some causal supersource transductions that RCC cannot compute. Experimental results on controlled sequences and on a real-world task involving chemical structures confirm the computational limitations of RCC, while assessing the efficiency and efficacy of CRCC in dealing both with pure causal and contextual prediction tasks. Moreover, results obtained for the real-world task show the superiority of the proposed approach versus RCC when exploring a task for which it is not known whether the structural causality assumption holds. [ABSTRACT FROM AUTHOR]

Abstract: Self-organizing models constitute valuable tools for data visualization, clustering, and data mining. Here, we focus on extensions of basic vector-based models by recursive computation in such a way that sequential and tree-structured data can be processed directly. The aim of this article is to give a unified review of important models recently proposed in literature, to investigate fundamental mathematical properties of these models, and to compare the approaches by experiments. We first review several models proposed in literature from a unifying perspective, thereby making use of an underlying general framework which also includes supervised recurrent and recursive models as special cases. We shortly discuss how the models can be related to different neuron lattices. Then, we investigate theoretical properties of the models in detail: we explicitly formalize how structures are internally stored in different context models and which similarity measures are induced by the recursive mapping onto the structures. We assess the representational capabilities of the models, and we shortly discuss the issues of topology preservation and noise tolerance. The models are compared in an experiment with time series data. Finally, we add an experiment for one context model for tree-structured data to demonstrate the capability to process complex structures. [Copyright &y& Elsevier]

Self-organization constitutes an important paradigm in machine learning with successful applications e.g. in data- and web-mining. Most approaches, however, have been proposed for processing data contained in a fixed and finite dimensional vector space. In this article, we will focus on extensions to more general data structures like sequences and tree structures. Various modifications of the standard self-organizing map (SOM) to sequences or tree structures have been proposed in the literature some of which are the temporal Kohonen map, the recursive SOM, and SOM for structured data. These methods enhance the standard SOM by utilizing recursive connections. We define a general recursive dynamic in this article which provides recursive processing of complex data structures by recursive computation of internal representations for the given context. The above mentioned mechanisms of SOMs for structures are special cases of the proposed general dynamic. Furthermore, the dynamic covers the supervised case of recurrent and recursive networks. The general framework offers an uniform notation for training mechanisms such as Hebbian learning. Moreover, the transfer of computational alternatives such as vector quantization or the neural gas algorithm to structure processing networks can be easily achieved. One can formulate general cost functions corresponding to vector quantization, neural gas, and a modification of SOM. The cost functions can be compared to Hebbian learning which can be interpreted as an approximation of a stochastic gradient descent. For comparison, we derive the exact gradients for general cost functions. [Copyright &y& Elsevier]

We propose a deep Graph Neural Network (GNN) model that alternates two types of layers. The first type is inspired by Reservoir Computing (RC) and generates new vertex features by iterating a non-linear map until it converges to a fixed point. The second type of layer implements graph pooling operations, that gradually reduce the support graph and the vertex features, and further improve the computational efficiency of the RC-based GNN. The architecture is, therefore, pyramidal. In the last layer, the features of the remaining vertices are combined into a single vector, which represents the graph embedding. Through a mathematical derivation introduced in this paper, we show formally how graph pooling can reduce the computational complexity of the model and speed-up the convergence of the dynamical updates of the vertex features. Our proposed approach to the design of RC-based GNNs offers an advantageous and principled trade-off between accuracy and complexity, which we extensively demonstrate in experiments on a large set of graph datasets. [ABSTRACT FROM AUTHOR]

The adaptive processing of graph data is a long-standing research topic that has been lately consolidated as a theme of major interest in the deep learning community. The snap increase in the amount and breadth of related research has come at the price of little systematization of knowledge and attention to earlier literature. This work is a tutorial introduction to the field of deep learning for graphs. It favors a consistent and progressive presentation of the main concepts and architectural aspects over an exposition of the most recent literature, for which the reader is referred to available surveys. The paper takes a top-down view of the problem, introducing a generalized formulation of graph representation learning based on a local and iterative approach to structured information processing. Moreover, it introduces the basic building blocks that can be combined to design novel and effective neural models for graphs. We complement the methodological exposition with a discussion of interesting research challenges and applications in the field. [ABSTRACT FROM AUTHOR]

Recurrent Neural Networks (RNNs) represent a natural paradigm for modeling sequential data like text written in natural language. In fact, RNNs and their variations have long been the architecture of choice in many applications, however in practice they require the use of labored architectures (such as gating mechanisms) and computationally heavy training processes. In this paper we address the question of whether it is possible to generate sentence embeddings via completely untrained recurrent dynamics, on top of which to apply a simple learning algorithm for text classification. This would allow to obtain extremely efficient models in terms of training time. Our work investigates the extent to which this approach can be used, by analyzing the results on different tasks. Finally, we show that, within certain limits, it is possible to build extremely efficient models for text classification that remain competitive in accuracy with reference models in the state-of-the-art. [ABSTRACT FROM AUTHOR]

We propose a novel methodology for representation learning on graph-structured data, in which a stack of Bayesian Networks learns different distributions of a vertex's neighbourhood. Through an incremental construction policy and layer-wise training, we can build deeper architectures with respect to typical graph convolutional neural networks, with benefits in terms of context spreading between vertices. First, the model learns from graphs via maximum likelihood estimation without using target labels. Then, a supervised readout is applied to the learned graph embeddings to deal with graph classification and vertex classification tasks, showing competitive results against neural models for graphs. The computational complexity is linear in the number of edges, facilitating learning on large scale data sets. By studying how depth affects the performances of our model, we discover that a broader context generally improves performances. In turn, this leads to a critical analysis of some benchmarks used in literature. Keywords: Structured domains, deep graph networks, graph neural networks, deep learning, maximum likelihood, graph classification, node classification. [ABSTRACT FROM AUTHOR]

The increasing digitization and datification of all aspects of people's daily life, and the consequent growth in the use of personal data, are increasingly challenging the current development and adoption of Machine Learning (ML). First, the sheer complexity and amount of data available in these applications strongly demands for ML algorithms that can be trained directly on complex structures, which can be naturally described by graphs. In fact, graphs inherently capture information about entities, their attributes, and relationships between them. Directly applying ML to graphs relieves domain experts and data scientists from the challenging and time-consuming problem of designing a suitable vector-based data representation used by classical ML techniques. Second, ML algorithms should not only be designed to achieve high technical and functional standards; as the automated decisions provided by these algorithms can have a relevant impact on people's lives, their behavior has to be aligned with the values and principles of individuals and society. This demands for designing automated algorithms that we, as humans, can trust, fulfilling the requirements of fairness, robustness, privacy, and explainability. Third, designing effective ML algorithms requires skills and expertise developed at different levels. This substantially hinders the democratization and widespread availability of such technology for society at large, which in turn demands for improving the level of automatization and systematization of their design process, while also providing guarantees on their performance. For this reason, this paper provides an overview of the current works focused towards learning trustworthily, automatically, and with guarantees on graphs. [ABSTRACT FROM AUTHOR]

The goal of this research is to recognize the nest digging activity of tortoises using a device mounted atop the tortoise carapace. The device classifies tortoise movements in order to discriminate between nest digging, and non-digging activity (specifically walking and eating). Accelerometer data was collected from devices attached to the carapace of a number of tortoises during their two-month nesting period. Our system uses an accelerometer and an activity recognition system (ARS) which is modularly structured using an artificial neural network and an output filter. For the purpose of experiment and comparison, and with the aim of minimizing the computational cost, the artificial neural network has been modelled according to three different architectures based on the input delay neural network (IDNN). We show that the ARS can achieve very high accuracy on segments of data sequences, with an extremely small neural network that can be embedded in programmable low power devices. Given that digging is typically a long activity (up to two hours), the application of ARS on data segments can be repeated over time to set up a reliable and efficient system, called Tortoise@, for digging activity recognition. [ABSTRACT FROM AUTHOR]

In this paper we address the problem of one day-ahead hourly electricity price forecast for smart grid applications. To this aim, we investigate the application of a number of predictive models for time-series, including methods based on empirical strategies frequently adopted in the smart grid community, Kalman Filters and Echo State Networks (ESNs). The considered methods have been suitably modified to address the electricity price forecast problem. Strategies based on daily re-adaptation of models’ parameters are taken into consideration as well. The predictive performance achieved by the considered models is assessed, and the methods are compared among each other on recent real data from the Italian electricity market. As a result of the comparison over three years data, ESN methods appear to provide the most accurate price predictions, which could imply significant economic savings in many smart grid activities, such as switching on power plants to support power generation from renewable sources, electric vehicle recharging or usage of household appliances. [ABSTRACT FROM AUTHOR]

Real-time user localization in indoor environments is an important issue in ambient assisted living (AAL). In this context, localization based on received signal strength (RSS) has received considerable interest in the recent literature, due to its low cost and energy consumption and to its availability on all wireless communication hardware. On the other hand, the RSS-based localization is characterized by a greater error with respect to other technologies. Restricting the problem to localization of AAL users in indoor environments, we demonstrate that forecasting with a little user movement advance (for example, when the user is about to leave a room) provides significant benefits to the accuracy of RSS-based localization systems. Specifically, we exploit echo state networks (ESNs) fed with RSS measurements and trained to recognize patterns of user's movements to feed back to the RSS-based localization system. [ABSTRACT FROM AUTHOR]

Abstract: This paper reports some recent results from the empirical evaluation of different types of structured molecular representations used in QSPR analysis through a recursive neural network (RNN) model, which allows for their direct use without the need for measuring or computing molecular descriptors. This RNN methodology has been applied to the prediction of the properties of small molecules and polymers. In particular, three different descriptions of cyclic moieties, namely group, template and cycle break have been proposed. The effectiveness of the proposed method in dealing with different representations of chemical structures, either specifically designed or of more general use, has been demonstrated by its application to data sets encompassing various types of cyclic structures. For each class of experiments a test set with data that were not used for the development of the model was used for validation, and the comparisons have been based on the test results. The reported results highlight the flexibility of the RNN in directly treating different classes of structured input data without using input descriptors. [Copyright &y& Elsevier]

Abstract: A recursive neural network QSPR model that can take directly molecular structures as input was applied to the prediction of the glass transition temperature of 277 poly(meth)acrylates. This model satisfactorily predicted the chemical–physical properties of high and low molecular weight acyclic compounds. However, side-chain benzene rings are present in about one half of the selected polymers. In order to render cyclic structures, the molecular representation through hierarchical structures was extended by two methods, named group and cycle breaking, respectively. The latter approach exploits standard unique molecular description systems, i.e. Unique SMILES and InChI. In all cases the prediction was very good, with 15–16K mean absolute error and 19–21K standard deviation. This result confirms the robustness of our method with respect to the inclusion of different structures. Moreover, the good performance of the cycle breaking representation paves the way for the investigation of data sets that contain a variety of poorly sampled cyclic structures. [Copyright &y& Elsevier]

Smart robotic environments combine traditional (ambient) sensing devices and mobile robots. This combination extends the type of applications that can be considered, reduces their complexity, and enhances the individual values of the devices involved by enabling new services that cannot be performed by a single device. To reduce the amount of preparation and preprogramming required for their deployment in real‐world applications, it is important to make these systems self‐adapting. The solution presented in this paper is based upon a type of compositional adaptation where (possibly multiple) plans of actions are created through planning and involve the activation of pre‐existing capabilities. All the devices in the smart environment participate in a pervasive learning infrastructure, which is exploited to recognize which plans of actions are most suited to the current situation. The system is evaluated in experiments run in a real domestic environment, showing its ability to proactively and smoothly adapt to subtle changes in the environment and in the habits and preferences of their user(s), in presence of appropriately defined performance measuring functions. [ABSTRACT FROM AUTHOR]

Estimation of mortality risk of very preterm neonates is carried out in clinical and research settings. We aimed at elaborating a prediction tool using machine learning methods. We developed models on a cohort of 23747 neonates <30 weeks gestational age, or <1501 g birth weight, enrolled in the Italian Neonatal Network in 2008-2014 (development set), using 12 easily collected perinatal variables. We used a cohort from 2015-2016 (N = 5810) as a test set. Among several machine learning methods we chose artificial Neural Networks (NN). The resulting predictor was compared with logistic regression models. In the test cohort, NN had a slightly better discrimination than logistic regression (P < 0.002). The differences were greater in subgroups of neonates (at various gestational age or birth weight intervals, singletons). Using a cutoff of death probability of 0.5, logistic regression misclassified 67/5810 neonates (1.2 percent) more than NN. In conclusion our study - the largest published so far - shows that even in this very simplified scenario, using only limited information available up to 5 minutes after birth, a NN approach had a small but significant advantage over current approaches. The software implementing the predictor is made freely available to the community. [ABSTRACT FROM AUTHOR]

The objective of this work is the development of a learning system for the automatic assessment of balance abilities in elderly people. The system is based on estimating the Berg Balance Scale (BBS) score from the stream of sensor data gathered by a Wii Balance Board. The scientific challenge tackled by our investigation is to assess the feasibility of exploiting the richness of the temporal signals gathered by the balance board for inferring the complete BBS score based on data from a single BBS exercise. The relation between the data collected by the balance board and the BBS score is inferred by neural networks for temporal data, modeled in particular as Echo State Networks within the Reservoir Computing (RC) paradigm, as a result of a comprehensive comparison among different learning models. The proposed system results to be able to estimate the complete BBS score directly from temporal data on exercise #10 of the BBS test, with ≈ 10 s of duration. Experimental results on real-world data show an absolute error below 4 BBS score points (i.e. below the 7% of the whole BBS range), resulting in a favorable trade-off between predictive performance and user’s required time with respect to previous works in literature. Results achieved by RC models compare well also with respect to different related learning models. Overall, the proposed system puts forward as an effective tool for an accurate automated assessment of balance abilities in the elderly and it is characterized by being unobtrusive, easy to use and suitable for autonomous usage. [ABSTRACT FROM AUTHOR]

Robotic ecologies are systems made out of several robotic devices, including mobile robots, wireless sensors and effectors embedded in everyday environments, where they cooperate to achieve complex tasks. This paper demonstrates how endowing robotic ecologies with information processing algorithms such as perception, learning, planning, and novelty detection can make these systems able to deliver modular, flexible, manageable and dependable Ambient Assisted Living (AAL) solutions. Specifically, we show how the integrated and self-organising cognitive solutions implemented within the EU project RUBICON (Robotic UBIquitous Cognitive Network) can reduce the need of costly pre-programming and maintenance of robotic ecologies. We illustrate how these solutions can be harnessed to (i) deliver a range of assistive services by coordinating the sensing & acting capabilities of heterogeneous devices, (ii) adapt and tune the overall behaviour of the ecology to the preferences and behaviour of its inhabitants, and also (iii) deal with novel events, due to the occurrence of new user׳s activities and changing user׳s habits. [ABSTRACT FROM AUTHOR]

In this paper, we present an introduction and critical experimental evaluation of a reservoir computing (RC) approach for ambient assisted living (AAL) applications. Such an empirical analysis jointly addresses the issues of efficiency, by analyzing different system configurations toward the embedding into computationally constrained wireless sensor devices, and of efficacy, by analyzing the predictive performance on real-world applications. First, the approach is assessed on a validation scheme where training, validation and test data are sampled in homogeneous ambient conditions, i.e., from the same set of rooms. Then, it is introduced an external test set involving a new setting, i.e., a novel ambient, which was not available in the first phase of model training and validation. The specific test-bed considered in the paper allows us to investigate the capability of the RC approach to discriminate among user movement trajectories from received signal strength indicator sensor signals. This capability can be exploited in various AAL applications targeted at learning user indoor habits, such as in the proposed indoor movement forecasting task. Such a joint analysis of the efficiency/efficacy trade-off provides novel insight in the concrete successful exploitation of RC for AAL tasks and for their distributed implementation into wireless sensor networks. [ABSTRACT FROM AUTHOR]

With the increasing amount of information in electronic form the fields of Machine Learning and Data Mining continue to grow by providing new advances in theory, applications and systems. The aim of this paper is to consider some recent theoretical aspects and approaches to ML and DM with an emphasis on the Italian research. [ABSTRACT FROM AUTHOR]

Abstract: The suitability of the comet assay for quantifying DNA repair capacity at individual level was studied following the kinetics of nucleotide excision repair (NER) in human lymphocytes from four healthy donors, at various time steps after a single dose of UVC. A significant increase of DNA migration was seen as soon as 20min after UV exposure, reaching the peak within 60–90min. Afterwards, a rapid decline was observed, approaching the basal level at 180–240min. The increase could be ascribed to excision activity, while the reduction to gap filling and rejoining, as demonstrated by the effects of phase-specific inhibitors, novobiocin and aphidicolin. Therefore, the comet assay should allow following the biphasic kinetics of NER. Wide inter-individual differences were observed, although repeated tests on the same donor cells revealed a large experimental variation. To quantitatively compare the individual patterns, a mathematical model was developed that adequately fitted the experimental results and estimated appropriate descriptors for each phase and for each donor. A second approach was also used to directly compare the distributions of damaged cells and to assess the differences between donors and between experiments visualizing them as reciprocal distances on a two-dimensional space computed with a principal component analysis (PCA). The results confirmed the inter-individual differences, but also the strong influence of experimental factors of the comet assay. The two approaches provided the means of accurately comparing DNA repair kinetics at individual level, taking also into account the experimental variability which poses serious doubts on the suitability of the comet assay. Nevertheless, since this methodology allows a detailed analysis of repair kinetics and it is potentially very useful for identifying individual with reduced repair capacity, further efforts have to be addressed to improve the reproducibility of the comet assay. [Copyright &y& Elsevier]