Search

Abstract: Recurrent Neural Networks (RNNs) are typically used to learn representations in partially observable environments. Unfortunately, training RNNs is known to be difficult, and the difficulty increases for agents who learn online and continually interact with the environment. Two common strategies to overcome this difficulty are to approximate gradient-based algorithms for learning the recurrent state or to find a recurrent architecture for which a computationally cheap gradient-based learning algorithm exists. Methods in the second category often limit representational capacity, just as using linear activations or diagonal weight matrices. In this work, we propose a novel recurrent architecture called Recurrent Trace Units (RTUs). RTUs expand representation capacity, but remain inexpensive to train. We derive RT2, a real-time recurrent learning algorithm for RTUs that is tractable, exact, and has linear compute and memory complexities. We investigate performance on a diagnostic benchmark inspired by animal learning and across several partially observable control environments. We show the agents that use RT2 achieve overall better performance when faced with long-term prediction tasks and reach their goals faster in control tasks.

When applying machine learning to clinical practice, a major hurdle that will be encountered is the lack of available data. While the data collected in clinical therapies is suitable for the types of analysis that are needed to measure and track clinical outcomes, it may not be suitable for other types of analysis. For instance, video data may have poor alignment with behavioral data, making it impossible to extract the videos frames that directly correlate with the observed behavior. Alternatively, clinicians may be exploring new data modalities, such as physiological signal collection, to research methods of improving clinical outcomes that are incompatible with their existing tools. Both problems warrant the exploration of improving the tools available to clinicians and developing them in a way that allows future customization for future research and clinical needs. This thesis covers the development and user perceived usability of a custom software tool, called cometrics, that was designed to address this data gap and be accessible enough to cover future use cases. Various use cases of this software are explored and a survey from existing users was conducted to provide comparison to existing tools. Additionally, the free and open-source nature of the software not only ensures confidence in the handling of private health information, but also allows anyone to inspect and modify the source code for their specific needs. After deploying this software tool to two independent clinical spaces within the Munroe-Meyer Institute at the University of Nebraska Medical Center, two novel datasets were collected. A physiological dataset focused on expressions of emotion in children was analyzed using statistical models and benchmark machine learning techniques. Secondly, a video-based dataset focused on the expression of severe behavior in children was used for detecting instances of hitting. Both datasets demonstrate the rudimentary capability to train machine learning models to automate annotation of clinical data for both efficiency and early warning use cases. These advances are accelerated by the intersection of clinical practice and engineering, which is made easier using a tool that is made for both parties. Advisers: Benjamin Riggan and James Gehringer

Episodic memory is the memory of complex sequences of events; basically memories that contain information about ‘what’ occurred, ‘where’ it happened, and ‘when’. It is unclear how episodic memory is stored in the brain. The first part of this dissertation presents a model whereby the recurrent connections of hippocampal field CA3 are used to store a cue while tracking its appearance in time to allow the trace of temporally separated cues to be bound by long term potentiation (LTP). This process allows disparate elements of an episode to be linked and recovered in correct order while maintaining their temporal relationships. The proposed activities of field CA3 evident in the model were then validated by electrophysiological experiments showing that the CA3 network can produce long and broad network reverberations in vitro without the reverberation of individual pyramidal cells. Additional studies demonstrated that CA3 was required for storage of the ‘when’ component of episodic memory. Episodic memory storage and organization happens continuously without supervision; however animals and humans use past experience to organize incoming complex information. Research described in the second part of the dissertation used exploration and learning of a complex unsupervised environment to test if prior experience with environmental complexity influenced exploration strategies and learning. We found that rats spontaneously organize their behavior into episodes and that rats with prior experience with complexity use these episodes to more efficiently explore and learn the environment as compared to rats with prior exercise or handling. Using this behavioral task we then determined where learning-related synaptic modifications occurred. Analysis of the distribution of synapses with evidence of recent LTP showed that ‘prior experience’ animals store their information in spatially discrete segments of hippocampus, and primarily in field CA1. We then used a second behavioral task to determine where episodic vs contingency-based learning was stored. We found that in the exploration paradigm, synaptic changes associated with exploration were prominent in a different collection of zones including CA1 striatum oriens and CA3c. These mapping studies reinforce the conclusion that different types or components of memory are encoded through activities of different hippocampal subfields.

While the Chilean constitution guarantees the right of a clean environment and that environmental acts and policies to manage waste have been passed, waste generation has increased dramatically in the last decade in Chile and programmes to recycle, recover or reuse waste are not being implemented. The extent of Chile's waste management problem is vast. Among the existing problems for implementing waste management programmes in Chile is the lack of information on factors contributing to waste generation and the absence of waste generation forecasts. Recognising these waste generation factors is essential for implementing policies to reduce waste generation and waste generation forecasts are fundamental for planning waste management systems. This research aims to design an analysis tool to assess waste generating factors and forecast waste generation for a significant portion of Chile. Data for many variables indicating socio-demographic, economic, geographic and waste-related conditions were collected based on the existing literature. Using these variables, statistical methods identified Population, Percentage of Urban Population, Years of Education, Number of Libraries and Number of Indigents as the most important factors contributing to waste generation in Chile. A Multi-Layer Perceptron neural network modelled the relationship between these variables and waste generation with great accuracy (R² = .819). The MLP network determined their respective contribution to waste generation and showed that they all contribute positively to waste generation. Using these variables, a Self-Organising Feature Map neural network clustered the 342 communes of Chile into three groups (with 91, 156 and 95 communes) from which representative communes were selected for data collection for forecasting waste generation. The most representative communes were not used due to lack of data. Therefore, secondary representative communes were selected, reducing the level of representativeness of the model from 230 (67.3%) communes to 167 (48.8%). Data was collected from the secondary communes and forecasts for waste generation up to the year 2010 were made. Recurrent networks were the best neural networks for forecasting waste generation using the selected variables for the three groups (R² = 0.75, 0.25 and 0.80, respectively). These results were improved using Multi Layer Perceptrons and recurrent networks with Per Capita Waste Generation as a new input (R² = 0.81, 0.91 and 0.98), showing extremely accurate forecasts for the validation periods. Forecasted rates show that by 2010, representative communes will generate 100, 240 and 2,900 tonnes/month, reaching annual rates of 1%, 0.6% and -3%, respectively. The forecasted results were used to obtain estimates for the represented communes of each group. Total waste generation from the represented communes will peak at 3,800 tonnes/month and 18,500 tonnes/month by 2010 and over 330,000 tonnes/month by 2007. Extrapolating these results shows that Chile will peak at more than 500,000 tonnes/month by 2007, an increase of 7.6% in total waste generation from 2002. Finally, it has been demonstrated that artificial neural networks have the potential to work with waste data to great accuracy despite the problems with the data. The proposed model represents a reliable tool for improving waste management not only for Chile, but also abroad.