Search

Human and non-human primate studies clearly implicate the dorsolateral prefrontal cortex (dlPFC) as critical for advanced cognitive functions. 1,2 It is thought that intracortical synaptic architectures within the dlPFC are the integral neurobiological substrate that gives rise to these processes. 3,4,5,6,7 In the prevailing model, each cortical column makes up one fundamental processing unit composed of dense intrinsic connectivity, conceptualized as the "canonical" cortical microcircuit. 3,8 Each cortical microcircuit receives sensory and cognitive information from upstream sources, which are represented by sustained activity within the microcircuit, referred to as persistent or recurrent activity. 4,9 Via recurrent connections within the microcircuit, activity propagates for a variable length of time, thereby allowing temporary storage and computations to occur locally before ultimately passing a transformed representation to a downstream output. 4,5,10 Competing theories regarding how microcircuit activity is coordinated have proven difficult to reconcile in vivo , where intercortical and intracortical computations cannot be fully dissociated. 5,9,11,12 Here, using high-density calcium imaging of macaque dlPFC, we isolated intracortical computations by interrogating microcircuit networks ex vivo. Using peri-sulcal stimulation to evoke recurrent activity in deep layers, we found that activity propagates through stochastically assembled intracortical networks wherein orderly, predictable, low-dimensional collective dynamics arise from ensembles with highly labile cellular memberships. Microcircuit excitability covaried with individual cognitive performance, thus anchoring heuristic models of abstract cortical functions within quantifiable constraints imposed by the underlying synaptic architecture. Our findings argue against engram or localist architectures, together demonstrating that generation of high-fidelity population-level signals from distributed, labile networks is an intrinsic feature of dlPFC microcircuitry. [Display omitted] • dlPFC intrinsically maintains recurrent activity in the absence of extrinsic inputs • Identical inputs are routed through labile, stochastically assembled ensembles • Stable population-level representations arise from unstable ensembles • Microcircuit excitability covaried with individual cognitive flexibility Using ex vivo Ca2+ imaging in macaque dorsolateral prefrontal cortex, Nolan et al. reveal that recurrent activity propagates through unstable, stochastically assembled ensembles to produce stable population-level events. Network excitability covaried with set-shifting performance, suggesting that these features may underlie dlPFC's role in cognition. [ABSTRACT FROM AUTHOR]

A novel method of portfolio selection is provided with further higher moments, filtering with fundamentals in intelligent computing resources. The Evolutional Portfolio Optimization System (EPOS) evaluates unobtrusive relations from a vast amount of accounting and financial data, excluding hoax and noise, to select the optimal portfolio. The fundamental question of Free Will, limited in investment selection, is answered through a new philosophical approach. [ABSTRACT FROM AUTHOR]

Cortical networks exhibit complex stimulus-response patterns that are based on specific recurrent interactions between neurons. For example, the balance between excitatory and inhibitory currents has been identified as a central component of cortical computations. However, it remains unclear how the required synaptic connectivity can emergeindevelopingcircuitswheresynapsesbetweenexcitatoryandinhibitoryneurons are simultaneously plastic. Using theory and modeling, we propose that a wide range of cortical response properties can arise from a single plasticity paradigm that acts simultaneously at all excitatory and inhibitory connections--Hebbian learning that is stabilized by the synapse-type-specific competition for a limited supply of synaptic resources. In plastic recurrent circuits, this competition enables the formation and decorrelation of inhibition-balanced receptive fields. Networks develop an assembly structure with stronger synaptic connections between similarly tuned excitatory and inhibitory neurons and exhibit response normalization and orientation-specific centersurround suppression, reflecting the stimulus statistics during training. These results demonstrate how neurons can self-organize into functional networks and suggest an essential role for synapse-type-specific competitive learning in the development of cortical circuits. [ABSTRACT FROM AUTHOR]

Creating symbolic melodies with machine learning is challenging because it requires an understanding of musical structure and the handling of inter-dependencies and long-term dependencies. Learning the relationship between events that occur far apart in time in music poses a considerable challenge for machine learning models. Another notable feature of music is that notes must account for several inter-dependencies, including melodic, harmonic, and rhythmic aspects. Baseline methods, such as RNNs, LSTMs, and GRUs, often struggle to capture these dependencies, resulting in the generation of musically incoherent or repetitive melodies. As such, in this study, a hierarchical multi-head attention LSTM model is proposed for creating polyphonic symbolic melodies. This enables our model to generate more complex and expressive melodies than previous methods, while still being musically coherent. The model allows learning of long-term dependencies at different levels of abstraction, while retaining the ability to form inter-dependencies. The study has been conducted on two major symbolic music datasets, MAESTRO and Classical-Music MIDI, which feature musical content encoded on MIDI. The artistic nature of music poses a challenge to evaluating the generated content and qualitative analysis are often not enough. Thus, human listening tests are conducted to strengthen the evaluation. Qualitative analysis conducted on the generated melodies shows significantly improved loss scores on MSE over baseline methods, and is able to generate melodies that were both musically coherent and expressive. The listening tests conducted using Likert-scale support the qualitative results and provide better statistical scores over baseline methods. [ABSTRACT FROM AUTHOR]

Speaker-verification-system (SVS) is automated nowadays to improve the authenticity score of digital applications. However, spoofs in the audio signal have reduced the integrity score of the audio signal, which has tended to cause less authentication exactness score. Considering this, spoof recognition objectives emerged in this field to find the different types of spoofs with high exactness scores. Attracting the widest spoof forecasting score is impossible due to harmful and different spoof features. So, the present study built a novel Dove-based Recurrent Spoof Recognition System (DbRSRS) to identify the spoofing behaviour and its types from the trained audio data. The noise features were filtered in the primary stage to mitigate the complexity of spoof recognition. Moreover, the noise features filtered data is taken to the classification phase for feature selection and spoof recognition. Here, the spoof types were classified based on the different class features. Once the Spoof is identified, it is specified under different spoof classes. Here, the optimal dove features are utilized to tune the DbRSRS classification parameters. This process helped to earn the finest spoof recognition score than the recently published associated model. Henceforth, the recorded highest spoof forecasting accuracy was 99.2%, and the reported less error value was 0.05%. Thus, attaining the highest spoof prediction exactness score with less error value might improve the SVS performance. [ABSTRACT FROM AUTHOR]

Sea surface temperature (SST) is a key indicator of the global climate system and is directly related to marine and coastal ecosystems, weather conditions, and atmospheric events. Marine heat waves (MHWs), characterized by prolonged periods of high SST, affect significantly the oceanic water quality and thus, the local ecosystem, and marine and coastal activities. Given the anticipated increase of MHWs occurrences due to climate change, developing targeted strategies is needed to mitigate their impact. Accurate SST forecasting can significantly contribute to this cause and thus it comprises a crucial, yet challenging, task for the scientific community. Despite the wide variety of existing methods in the literature, the majority of them focus either on providing near-future SST forecasts (a few days until 1 month) or long-term predictions (decades to century) in climate scales based on hypothetical scenarios that need to be proven. In this work, we introduce a robust deep learning-based method for efficient SST forecasting of the interim future (1 year ahead) using high-resolution satellite-derived SST data. Our approach processes daily SST sequences lasting 1 year, along with five other relevant atmospheric variables, to predict the corresponding daily SST timeseries for the subsequent year. The novel method was deployed to accurately forecast SST over the northeastern Mediterranean Seas (Aegean, Ionian, Cretan Seas: AICS). Utilizing the effectiveness of well-established deep learning architectures, our method can provide accurate spatiotemporal predictions for multiple areas at once, without the need to be deployed separately at each sub-region. The modular design of the framework allows customization for different spatial and temporal resolutions according to use case requirements. The proposed model was trained and evaluated using available data from the AICS region over a 15-year time period (2008–2022). The results demonstrate the efficiency of our method in predicting SST variability, even for previously unseen data that are over 2 years in advance, in respect to the training set. The proposed methodology is a valuable tool that also can contribute to MHWs prediction. [ABSTRACT FROM AUTHOR]

People with hearing disabilities often face communication barriers when interacting with hearing individuals. To address this issue, this paper proposes a bidirectional Sign Language Translation System that aims to bridge the communication gap. Deep learning models such as recurrent neural networks (RNN), bidirectional RNN (BRNN), LSTM, GRU, and Transformers are compared to find the most accurate model for sign language recognition and translation. Keypoint detection using MediaPipe is employed to track and understand sign language gestures. The system features a user-friendly graphical interface with modes for translating between Mexican Sign Language (MSL) and Spanish in both directions. Users can input signs or text and obtain corresponding translations. Performance evaluation demonstrates high accuracy, with the BRNN model achieving 98.8% accuracy. The research emphasizes the importance of hand features in sign language recognition. Future developments could focus on enhancing accessibility and expanding the system to support other sign languages. This Sign Language Translation System offers a promising solution to improve communication accessibility and foster inclusivity for individuals with hearing disabilities. [ABSTRACT FROM AUTHOR]

Video enhancement is a challenging problem, more than that of stills, mainly due to high computational cost, larger data volumes and the difficulty of achieving consistency in the spatio-temporal domain. In practice, these challenges are often coupled with the lack of example pairs, which inhibits the application of supervised learning strategies. To address these challenges, we propose an efficient adversarial video enhancement framework that learns directly from unpaired video examples. In particular, our framework introduces new recurrent cells that consist of interleaved local and global modules for implicit integration of spatial and temporal information. The proposed design allows our recurrent cells to efficiently propagate spatio-temporal information across frames and reduces the need for high complexity networks. Our setting enables learning from unpaired videos in a cyclic adversarial manner, where the proposed recurrent units are employed in all architectures. Efficient training is accomplished by introducing one single discriminator that learns the joint distribution of source and target domain simultaneously. The enhancement results demonstrate clear superiority of the proposed video enhancer over the state-of-the-art methods, in all terms of visual quality, quantitative metrics, and inference speed. Notably, our video enhancer is capable of enhancing over 35 frames per second of FullHD video (1080x1920). [ABSTRACT FROM AUTHOR]

This research focuses on employing Recurrent Neural Networks (RNN) to prognosis a wind turbine operation’s health from collected vibration time series data, by using several memory cell variations, including Long Short Time Memory (LSTM), Bilateral LSTM (BiLSTM), and Gated Recurrent Unit (GRU), which are integrated into various architectures. We tune the training hyperparameters as well as the adapted depth and recurrent cell number of the proposed networks to obtain the most accurate predictions. Tuning those parameters is a hard task and depends widely on the experience of the designer. This can be resolved by integrating the training process in a Bayesian optimization loop where the loss is considered as the objective function to minimize. The obtained results show the effectiveness of the proposed method, which generates more accurate recurrent models with a more accurate prognosis of the operating state of the wind turbine than those generated using trivial training parameters.

People with hearing disabilities often face communication barriers when interacting with hearing individuals. To address this issue, this paper proposes a bidirectional Sign Language Translation System that aims to bridge the communication gap. Deep learning models such as recurrent neural networks (RNN), bidirectional RNN (BRNN), LSTM, GRU, and Transformers are compared to find the most accurate model for sign language recognition and translation. Keypoint detection using MediaPipe is employed to track and understand sign language gestures. The system features a user-friendly graphical interface with modes for translating between Mexican Sign Language (MSL) and Spanish in both directions. Users can input signs or text and obtain corresponding translations. Performance evaluation demonstrates high accuracy, with the BRNN model achieving 98.8% accuracy. The research emphasizes the importance of hand features in sign language recognition. Future developments could focus on enhancing accessibility and expanding the system to support other sign languages. This Sign Language Translation System offers a promising solution to improve communication accessibility and foster inclusivity for individuals with hearing disabilities.

The brain is targeted for processing temporal sequence information. It remains largely unclear how the brain learns to store and retrieve sequence memories. Here, we study how recurrent networks of binary neurons learn sequence attractors to store predefined pattern sequences and retrieve them robustly. We show that to store arbitrary pattern sequences, it is necessary for the network to include hidden neurons even though their role in displaying sequence memories is indirect. We develop a local learning algorithm to learn sequence attractors in the networks with hidden neurons. The algorithm is proven to converge and lead to sequence attractors. We demonstrate that the network model can store and retrieve sequences robustly on synthetic and real-world datasets. We hope that this study provides new insights in understanding sequence memory and temporal information processing in the brain. [ABSTRACT FROM AUTHOR]

The application of neural network models to scientific machine learning tasks has proliferated in recent years. In particular, neural networks have proved to be adept at modeling processes with spatial–temporal complexity. Nevertheless, these highly parameterized models have garnered skepticism in their ability to produce outputs with quantified error bounds over the regimes of interest. Hence there is a need to find uncertainty quantification methods that are suitable for neural networks. In this work we present comparisons of the parametric uncertainty quantification of neural networks modeling complex spatial–temporal processes with Hamiltonian Monte Carlo and Stein variational gradient descent and its projected variant. Specifically we apply these methods to graph convolutional neural network models of evolving systems modeled with recurrent neural network and neural ordinary differential equations architectures. We show that Stein variational inference is a viable alternative to Monte Carlo methods with some clear advantages for complex neural network models. For our exemplars, Stein variational interference gave similar pushed forward uncertainty profiles through time compared to Hamiltonian Monte Carlo, albeit with generally more generous variance. Projected Stein variational gradient descent also produced similar uncertainty profiles to the non-projected counterpart, but large reductions in the active weight space were confounded by the stability of the neural network predictions and the convoluted likelihood landscape. [ABSTRACT FROM AUTHOR]

Cortical dynamics are organized over multiple anatomical and temporal scales. The mechanistic origin of the temporal organization and its contribution to cognition remain unknown. Here, we demonstrate the cause of this organization by studying a specific temporal signature (time constant and latency) of neural activity. In monkey frontal areas, recorded during flexible decisions, temporal signatures display specific area-dependent ranges, as well as anatomical and cell-type distributions. Moreover, temporal signatures are functionally adapted to behaviourally relevant timescales. Fine-grained biophysical network models, constrained to account for experimentally observed temporal signatures, reveal that after-hyperpolarization potassium and inhibitory GABA-B conductances critically determine areas’ specificity. They mechanistically account for temporal signatures by organizing activity into metastable states, with inhibition controlling state stability and transitions. As predicted by models, state durations non-linearly scale with temporal signatures in monkey, matching behavioural timescales. Thus, local inhibitory-controlled metastability constitutes the dynamical core specifying the temporal organization of cognitive functions in frontal areas.

Classifying haemodialysis sessions, on the basis of the evolution of specific clinical variables over time, allows the physician to identify patients that are being treated inefficiently, and that may need additional monitoring or corrective interventions. In this paper, we propose a deep learning approach to clinical time series classification, in the haemodialysis domain. In particular, we have defined two novel architectures, able to take advantage of the strengths of Convolutional Neural Networks and of Recurrent Networks. The novel architectures we introduced and tested outperformed classical mathematical classification techniques, as well as simpler deep learning approaches. In particular, combining Recurrent Networks with convolutional structures in different ways, allowed us to obtain accuracies above 81%, coupled with high values of the Matthews Correlation Coefficient (MCC), a parameter particularly suitable to assess the quality of classification when dealing with unbalanced classes-as it was our case. In the future we will test an extension of the approach to additional monitoring time series, aiming at an overall optimization of patient care. [ABSTRACT FROM AUTHOR]

Reinforcement Learning (RL) agents are commonly thought of as adaptive decision procedures. They work on input/output data streams called "states", "actions" and "rewards". Most current research about RL adaptiveness to changes works under the assumption that the streams signatures (i.e. arity and types of inputs and outputs) remain the same throughout the agent lifetime. As a consequence, natural situations where the signatures vary (e.g. when new data streams become available, or when others become obsolete) are not studied. In this paper, we relax this assumption and consider that signature changes define a new learning situation called Protean Learning (PL). When they occur, traditional RL agents become undefined, so they need to restart learning. Can better methods be developed under the PL view? To investigate this, we first construct a stream-oriented formalism to properly define PL and signature changes. Then, we run experiments in an idealized PL situation where input addition and deletion occur during the learning process. Results show that a simple PL-oriented method enables graceful adaptation of these arity changes, and is more efficient than restarting the process. [ABSTRACT FROM AUTHOR]

Intent recognition is a key component of any task-oriented conversational system. The intent recognizer can be used first to classify the user's utterance into one of several predefined classes (intents) that help to understand the user's current goal. Then, the most adequate response can be provided accordingly. Intent recognizers also often appear as a form of joint models for performing the natural language understanding and dialog management tasks together as a single process, thus simplifying the set of problems that a conversational system must solve. This happens to be especially true for frequently asked question (FAQ) conversational systems. In this work, we first present an exploratory analysis in which different deep learning (DL) models for intent detection and classification were evaluated. In particular, we experimentally compare and analyze conventional recurrent neural networks (RNN) and state-of-the-art transformer models. Our experiments confirmed that best performance is achieved by using transformers. Specifically, best performance was achieved by fine-tuning the so-called BETO model (a Spanish pretrained bidirectional encoder representations from transformers (BERT) model from the Universidad de Chile) in our intent detection task. Then, as the main contribution of the paper, we analyze the effect of inserting unseen domain words to extend the vocabulary of the model as part of the fine-tuning or domain-adaptation process. Particularly, a very simple word frequency cut-off strategy is experimentally shown to be a suitable method for driving the vocabulary learning decisions over unseen words. The results of our analysis show that the proposed method helps to effectively extend the original vocabulary of the pretrained models. We validated our approach with a selection of the corpus acquired with the Hispabot-Covid19 system obtaining satisfactory results. [ABSTRACT FROM AUTHOR]

Recently, removing rain streaks from a single image has attracted a lot of attention because rain streaks can severely degrade the perceptual quality of the image and cause many practical vision systems to fail. Single image deraining can be served as a pre-processing step to improve the performance of high-level vision tasks such as object detection and video surveillance. In this paper, we propose recurrent scale-guide networks for single image deraining. Although the multi-scale strategy has been successfully applied to many computer vision problems, the correlation between different scales has not been explored in most existing methods. To overcome this deficiency, we propose two types of scale-guide blocks and develop two combinations between the blocks. One type of scale-guide block is that small scale guides the large, and the other is that large scale guides the small. Moreover, we extend the single-stage deraining model to the multi-stage recurrent framework and introduce the Long Short-Term Memory (LSTM) to link every stage. Extensive experiments verify that the scale-guide manner boosts the deraining performance and the recurrent style improves the deraining results. Experimental results demonstrate that the proposed method outperforms other state-of-the-art deraining methods on three widely used datasets: Rain100H, Rain100L, and Rain1200. The source codes can be found at https://supercong94.wixsite.com/supercong94. [ABSTRACT FROM AUTHOR]

Multitemporal remote-sensing images play a key role as a source of information for automated crop mapping and monitoring. The spatial/spectral pattern evolution along time provides information about the dynamics of the crops and are very useful for productivity estimation. Although the multitemporal mapping of crops has progressed considerably with the advent of deep learning in recent years, the classification models obtained still have limitations when exposed to unknown classes in the prediction phase, reducing their usefulness. In other words, these models are trained to identify a closed set of crops (e.g., soy and sugar cane) and are therefore unable to recognize other types of crops (e.g., maize). In this letter, we deal with the challenges of multitemporal crop recognition by proposing a new approach called OpenPCS++ that is not only able to learn known classes but is also capable of identifying new crops in the predicting phase. The proposed approach was evaluated in two challenging public datasets located in tropical climates in Brazil. Results showed that OpenPCS++ achieved increases of up to 0.19 in terms of area under the receiver-operating characteristic (ROC) curve in comparison with baselines. Code is available at https://github.com/DiMorten/osss-mcr. [ABSTRACT FROM AUTHOR]

Summary: Despite ongoing experiential change, neural activity maintains remarkable stability. Although this is thought to be mediated by homeostatic plasticity, what aspect of neural activity is conserved and how the flexibility necessary for learning and memory is maintained is not fully understood. Experimental studies suggest that there exists network-centered, in addition to the well-studied neuron-centered, control. Here we computationally study such a potential mechanism: input-dependent inhibitory plasticity (IDIP). In a hippocampal model, we show that IDIP can explain the emergence of active and silent place cells as well as remapping following silencing of active place cells. Furthermore, we show that IDIP can also stabilize recurrent dynamics while preserving firing rate heterogeneity and stimulus representation, as well as persistent activity after memory encoding. Hence, the establishment of global network balance with IDIP has diverse functional implications and may be able to explain experimental phenomena across different brain areas.

Current concepts of sensory processing in the cerebral cortex emphasize serial extraction and recombination of features in hierarchically structured feed-forward networks in order to capture the relations among the components of perceptual objects. These concepts are implemented in convolutional deep learning networks and have been validated by the astounding similarities between the functional properties of artificial systems and their natural counterparts. However, cortical architectures also display an abundance of recurrent coupling within and between the layers of the processing hierarchy. This massive recurrence gives rise to highly complex dynamics whose putative function is poorly understood. Here a concept is proposed that assigns specific functions to the dynamics of cortical networks and combines, in a unifying approach, the respective advantages of recurrent and feed-forward processing. It is proposed that the priors about regularities of the world are stored in the weight distributions of feed-forward and recurrent connections and that the high-dimensional, dynamic space provided by recurrent interactions is exploited for computations. These comprise the ultrafast matching of sensory evidence with the priors covertly represented in the correlation structure of spontaneous activity and the context-dependent grouping of feature constellations characterizing natural objects. The concept posits that information is encoded not only in the discharge frequency of neurons but also in the precise timing relations among the discharges. Results of experiments designed to test the predictions derived from this concept support the hypothesis that cerebral cortex exploits the high-dimensional recurrent dynamics for computations serving predictive coding. [ABSTRACT FROM AUTHOR]

Multi-sensor distributed fusion estimation algorithms based on machine learning are proposed in this paper. Firstly, using local estimations as inputs and estimations of three classic distributed fusion (weighted by matrices, by diagonal matrices and by scalars) as the training sets, three distributed fusion algorithms based on BP network (BP net-based fusion weighted by matrices, by diagonal matrices and by scalar) are proposed and the selection basis of the number of nodes in hidden layer is given. Furthermore, by using local estimations as inputs and centralized fusion estimation as training set, another recurrent net-based distributed fusion algorithm is proposed, in the case that neither true states nor cross-covariance matrices is available. This method is not limited to the linear minimum variance (LMV) criterion, so its accuracy is higher than the classical three distributed fusion algorithms. A radar tracking simulation verifies the effectiveness of the proposed fusion networks.

Recurrent circuitry components are distributed widely within the brain, including both excitatory and inhibitory synaptic connections. Recurrent neuronal networks have potential stability problems, perhaps a predisposition to epilepsy. More generally, instability risks making internal representations of information unreliable. To assess the inherent stability properties of such recurrent networks, we tested a linear summation, non-spiking neuron model with and without a “dynamic leak”, corresponding to the low-pass filtering of synaptic input current by the RC circuit of the biological membrane. We first show that the output of this neuron model, in either of its two forms, follows its input at a higher fidelity than a wide range of spiking neuron models across a range of input frequencies. Then we constructed fully connected recurrent networks with equal numbers of excitatory and inhibitory neurons and randomly distributed weights across all synapses. When the networks were driven by pseudorandom sensory inputs with varying frequency, the recurrent network activity tended to induce high frequency self-amplifying components, sometimes evident as distinct transients, which were not present in the input data. The addition of a dynamic leak based on known membrane properties consistently removed such spurious high frequency noise across all networks. Furthermore, we found that the neuron model with dynamic leak imparts a network stability that seamlessly scales with the size of the network, conduction delays, the input density of the sensory signal and a wide range of synaptic weight distributions. Our findings suggest that neuronal dynamic leak serves the beneficial function of protecting recurrent neuronal circuitry from the self-induction of spurious high frequency signals, thereby permitting the brain to utilize this architectural circuitry component regardless of network size or recurrency.

Recurrent circuitry components are distributed widely within the brain, including both excitatory and inhibitory synaptic connections. Recurrent neuronal networks have potential stability problems, perhaps a predisposition to epilepsy. More generally, instability risks making internal representations of information unreliable. To assess the inherent stability properties of such recurrent networks, we tested a linear summation, non-spiking neuron model with and without a "dynamic leak", corresponding to the low-pass filtering of synaptic input current by the RC circuit of the biological membrane. We first show that the output of this neuron model, in either of its two forms, follows its input at a higher fidelity than a wide range of spiking neuron models across a range of input frequencies. Then we constructed fully connected recurrent networks with equal numbers of excitatory and inhibitory neurons and randomly distributed weights across all synapses. When the networks were driven by pseudorandom sensory inputs with varying frequency, the recurrent network activity tended to induce high frequency self-amplifying components, sometimes evident as distinct transients, which were not present in the input data. The addition of a dynamic leak based on known membrane properties consistently removed such spurious high frequency noise across all networks. Furthermore, we found that the neuron model with dynamic leak imparts a network stability that seamlessly scales with the size of the network, conduction delays, the input density of the sensory signal and a wide range of synaptic weight distributions. Our findings suggest that neuronal dynamic leak serves the beneficial function of protecting recurrent neuronal circuitry from the self-induction of spurious high frequency signals, thereby permitting the brain to utilize this architectural circuitry component regardless of network size or recurrency. [ABSTRACT FROM AUTHOR]

Recurrent cortical networks provide reservoirs of states that are thought to play a crucial role for sequential information processing in the brain. However, classical reservoir computing requires manual adjustments of global network parameters, particularly of the spectral radius of the recurrent synaptic weight matrix. It is hence not clear if the spectral radius is accessible to biological neural networks. Using random matrix theory, we show that the spectral radius is related to local properties of the neuronal dynamics whenever the overall dynamical state is only weakly correlated. This result allows us to introduce two local homeostatic synaptic scaling mechanisms, termed flow control and variance control, that implicitly drive the spectral radius toward the desired value. For both mechanisms the spectral radius is autonomously adapted while the network receives and processes inputs under working conditions. We demonstrate the effectiveness of the two adaptation mechanisms under different external input protocols. Moreover, we evaluated the network performance after adaptation by training the network to perform a time-delayed XOR operation on binary sequences. As our main result, we found that flow control reliably regulates the spectral radius for different types of input statistics. Precise tuning is however negatively affected when interneural correlations are substantial. Furthermore, we found a consistent task performance over a wide range of input strengths/variances. Variance control did however not yield the desired spectral radii with the same precision, being less consistent across different input strengths. Given the effectiveness and remarkably simple mathematical form of flow control, we conclude that self-consistent local control of the spectral radius via an implicit adaptation scheme is an interesting and biological plausible alternative to conventional methods using set point homeostatic feedback controls of neural firing.

Recurrent cortical networks provide reservoirs of states that are thought to play a crucial role for sequential information processing in the brain. However, classical reservoir computing requires manual adjustments of global network parameters, particularly of the spectral radius of the recurrent synaptic weight matrix. It is hence not clear if the spectral radius is accessible to biological neural networks. Using random matrix theory, we show that the spectral radius is related to local properties of the neuronal dynamics whenever the overall dynamical state is only weakly correlated. This result allows us to introduce two local homeostatic synaptic scaling mechanisms, termed flow control and variance control, that implicitly drive the spectral radius toward the desired value. For both mechanisms the spectral radius is autonomously adapted while the network receives and processes inputs under working conditions. We demonstrate the effectiveness of the two adaptation mechanisms under different external input protocols. Moreover, we evaluated the network performance after adaptation by training the network to perform a time-delayed XOR operation on binary sequences. As our main result, we found that flow control reliably regulates the spectral radius for different types of input statistics. Precise tuning is however negatively affected when interneural correlations are substantial. Furthermore, we found a consistent task performance over a wide range of input strengths/variances. Variance control did however not yield the desired spectral radii with the same precision, being less consistent across different input strengths. Given the effectiveness and remarkably simple mathematical form of flow control, we conclude that self-consistent local control of the spectral radius via an implicit adaptation scheme is an interesting and biological plausible alternative to conventional methods using set point homeostatic feedback controls of neural firing. [ABSTRACT FROM AUTHOR]

Intent recognition is a key component of any task-oriented conversational system. The intent recognizer can be used first to classify the user’s utterance into one of several predefined classes (intents) that help to understand the user’s current goal. Then, the most adequate response can be provided accordingly. Intent recognizers also often appear as a form of joint models for performing the natural language understanding and dialog management tasks together as a single process, thus simplifying the set of problems that a conversational system must solve. This happens to be especially true for frequently asked question (FAQ) conversational systems. In this work, we first present an exploratory analysis in which different deep learning (DL) models for intent detection and classification were evaluated. In particular, we experimentally compare and analyze conventional recurrent neural networks (RNN) and state-of-the-art transformer models. Our experiments confirmed that best performance is achieved by using transformers. Specifically, best performance was achieved by fine-tuning the so-called BETO model (a Spanish pretrained bidirectional encoder representations from transformers (BERT) model from the Universidad de Chile) in our intent detection task. Then, as the main contribution of the paper, we analyze the effect of inserting unseen domain words to extend the vocabulary of the model as part of the fine-tuning or domain-adaptation process. Particularly, a very simple word frequency cut-off strategy is experimentally shown to be a suitable method for driving the vocabulary learning decisions over unseen words. The results of our analysis show that the proposed method helps to effectively extend the original vocabulary of the pretrained models. We validated our approach with a selection of the corpus acquired with the Hispabot-Covid19 system obtaining satisfactory results.

Trading off exploration and exploitation in an unknown environment is key to maximising expected online return during learning. A Bayes-optimal policy, which does so optimally, conditions its actions not only on the environment state but also on the agent's uncertainty about the environment. Computing a Bayes-optimal policy is however intractable for all but the smallest tasks. In this paper, we introduce variational Bayes-Adaptive Deep RL (variBAD), a way to meta-learn approximately Bayes-optimal policies for complex tasks. VariBAD simultaneously meta-learns a variational auto-encoder to perform approximate inference, and a policy that incorporates task uncertainty directly during action selection by conditioning on both the environment state and the approximate belief. In two toy domains, we illustrate how variBAD performs structured online exploration as a function of task uncertainty. We further evaluate variBAD on MuJoCo tasks widely used in meta-RL and show that it achieves higher online return than existing methods. On the recently proposed Meta-World ML1 benchmark, variBAD achieves state of the art results by a large margin, fully solving two out of the three ML1 tasks for the first time. [ABSTRACT FROM AUTHOR]

• A novel N - to - N recurrent network for pixel-wise crop labeling of image sequences. • It combines RNN, encoder-decoder architectures and prior knowledge on crop dynamics. • Experiments on multi-date SAR datasets representing complex spatio-temporal dynamics. Crop recognition in tropical regions is a challenging task because of the highly complex crop dynamics, with multiple crops per year. Nevertheless, most automatic methods proposed thus far are devoted to temperate areas where normally a single crop is cultivated along the crop year. This paper introduces convolutional recurrent networks for crop recognition in areas characterized by complex spatiotemporal dynamics typical of tropical agriculture, where a per date classification is required. The proposed networks consist of two sequential steps. First, a deep network simultaneously models spatial and temporal contexts. Second, a post-processing algorithm enforces prior knowledge about the crop dynamics in the target area based on the posterior probabilities computed in the first step. The paper proposes deep network architectures that join a fully convolutional network (FCN) for modeling spatial context at multiple levels and a bidirectional recurrent neural network to explore the temporal context. The recurrent network is configured as N - to - N , where N is the sequence length. This allows it to produce classification outcomes for the entire sequence of multi-temporal images using a single network. Different network designs are proposed based on three FCN architectures: U-Net, dense network, and Atrous Spatial Pyramid Pooling. A convolutional Long-Short-Term-Memory (ConvLSTM) accounts for sequence modeling, whereas the Most Likely Class Sequence (MLCS) algorithm is adopted for enforcing prior knowledge. The paper finally reports experiments conducted on Sentinel-1 data of two publicly available datasets from different tropical regions. The experimental results indicated that the proposed architectures outperformed state-of-the-art methods based on recurrent networks in terms of Overall Accuracy and per-class F1 score. [ABSTRACT FROM AUTHOR]

Sequential activity has been observed in multiple neuronal circuits across species, neural structures, and behaviors. It has been hypothesized that sequences could arise from learning processes. However, it is still unclear whether biologically plausible synaptic plasticity rules can organize neuronal activity to form sequences whose statistics match experimental observations. Here, we investigate temporally asymmetric Hebbian rules in sparsely connected recurrent rate networks and develop a theory of the transient sequential activity observed after learning. These rules transform a sequence of random input patterns into synaptic weight updates. After learning, recalled sequential activity is reflected in the transient correlation of network activity with each of the stored input patterns. Using mean-field theory, we derive a low-dimensional description of the network dynamics and compute the storage capacity of these networks. Multiple temporal characteristics of the recalled sequential activity are consistent with experimental observations. We find that the degree of sparseness of the recalled sequences can be controlled by nonlinearities in the learning rule. Furthermore, sequences maintain robust decoding, but display highly labile dynamics, when synaptic connectivity is continuously modified due to noise or storage of other patterns, similar to recent observations in hippocampus and parietal cortex. Finally, we demonstrate that our results also hold in recurrent networks of spiking neurons with separate excitatory and inhibitory populations. [ABSTRACT FROM AUTHOR]

• Network models were used to study the possible functions of nonspecific top-down inputs. • Nonspecific top-down signals can suppress undesired bottom-up connections. • Synchronous oscillatory activity is essential for learning of bottom-up connections. • Inputs to excitatory and inhibitory neurons have distinct roles. In the brain, high-order and low-order areas are connected via bottom-up connections (from low-order to high-order areas) and top-down connections (from high-order to low-order areas). While bottom-up signals are thought to be critical in generating perception, functions of top-down signals have not been clearly delineated. One popular theory is that top-down inputs modify the activity of specific cell assemblies to modulate responses to bottom-up inputs. However, a different line of studies proposes that not all top-down inputs are specifically delivered. As the leading theories cannot account for nonspecific top-down inputs, we seek potential functions of nonspecific top-down signals using network models in our study. Our simulation results suggest that top-down inputs can regulate low-order area responses by providing temporal information even without spatial specificity. Specifically, the temporal information in nonspecific top-down inputs can weaken the undesired bottom-up connections, contributing to bottom-up connections' learning. Further, we found that cortical rhythms (synchronous oscillatory neural responses) are critical in the proposed learning process of bottom-up connections in our model. [ABSTRACT FROM AUTHOR]

Our decisions often depend on multiple sensory experiences separated by time delays. The brain can remember these experiences and, simultaneously, estimate the timing between events. To understand the mechanisms underlying working memory and time encoding, we analyze neural activity recorded during delays in four experiments on nonhuman primates. To disambiguate potential mechanisms, we propose two analyses, namely, decoding the passage of time from neural data and computing the cumulative dimensionality of the neural trajectory over time. Time can be decoded with high precision in tasks where timing information is relevant and with lower precision when irrelevant for performing the task. Neural trajectories are always observed to be low-dimensional. In addition, our results further constrain the mechanisms underlying time encoding as we find that the linear "ramping" component of each neuron's firing rate strongly contributes to the slow timescale variations that make decoding time possible. These constraints rule out working memory models that rely on constant, sustained activity and neural networks with high-dimensional trajectories, like reservoir networks. Instead, recurrent networks trained with backpropagation capture the time-encoding properties and the dimensionality observed in the data. [ABSTRACT FROM AUTHOR]

Indoor person tracking attracts a considerable effort from the research community as it allows to perform Human Behaviour Analysis tasks, where wireless technologies play a key role. However, complex signal propagation effects in indoor environments are the main issue to face when performing accurate indoor positioning and tracking. The advances in machine and deep learning models, applied to improve the estimation of the position captured by wireless sensors, can provide a more precise tracking and positioning, an open field for research which has been used to improve the prior art. In this paper, a novel framework for Adaptive Indoor Tracking using Recurrent models, in combination with Generative networks for new data generation (recovery), is presented (RecTrack-GAN). Firstly, a Received Signal Strength Indicator RSSI Fingerprinting database is collected. Secondly, a Recurrent Neural Network (RNN) takes as input the RSSI parameters collected by a Wireless Sensor Network (WSN) and estimates of both orientation and velocity using devices equipped with Inertial Measurement Unit (IMU) sensors, and learns to model the human movement based on these parameters. Thirdly, a Conditional Generative Adversarial Network (CGAN) is used to perform data recovering when no measurements are received and to update the Fingerprinting database taking into account the day time. The experiments performed showed that RecTrack-GAN improves accuracy performance and reduces error deviation for tracking up to 15% compared to the prior art in the literature. [ABSTRACT FROM AUTHOR]

The biomedical field is characterized by an ever-increasing production of sequential data, which often come in the form of biosignals capturing the time-evolution of physiological processes, such as blood pressure and brain activity. This has motivated a large body of research dealing with the development of machine learning techniques for the predictive analysis of such biosignals. Unfortunately, in high-stakes decision making, such as clinical diagnosis, the opacity of machine learning models becomes a crucial aspect to be addressed in order to increase the trust and adoption of AI technology. In this paper, we propose a model agnostic explanation method, based on occlusion, that enables the learning of the input’s influence on the model predictions. We specifically target problems involving the predictive analysis of time-series data and the models that are typically used to deal with data of such nature, i.e., recurrent neural networks. Our approach is able to provide two different kinds of explanations: one suitable for technical experts, who need to verify the quality and correctness of machine learning models, and one suited to physicians, who need to understand the rationale underlying the prediction to make aware decisions. A wide experimentation on different physiological data demonstrates the effectiveness of our approach both in classification and regression tasks.

Exo-atmospheric infrared (IR) point target discrimination is an important research topic of space surveillance systems. It is difficult to describe the characteristic information of the shape and micro-motion states of the targets and to discriminate different targets effectively by the characteristic information. This paper has constructed the infrared signature model of spatial point targets and obtained the infrared radiation intensity sequences dataset of different types of targets. This paper aims to design an algorithm for the classification problem of infrared radiation intensity sequences of spatial point targets. Recurrent neural networks (RNNs) are widely used in time series classification tasks, but face several problems such as gradient vanishing and explosion, etc. In view of shortcomings of RNNs, this paper proposes an independent random recurrent neural network (IRRNN) model, which combines independent structure RNNs with randomly weighted RNNs. Without increasing the training complexity of network learning, our model solves the problem of gradient vanishing and explosion, improves the ability to process long sequences, and enhances the comprehensive classification performance of the algorithm effectively. Experiments show that the IRRNN algorithm performs well in classification tasks and is robust to noise. [ABSTRACT FROM AUTHOR]

Action recognition plays an important role in various applications such as video monitoring, automatic video indexing, crowd analysis, human-machine interaction, smart homes and personal assistive robotics. In this paper, we propose improvements to some methods for human action recognition from videos that work with data represented in the form of skeleton poses. These methods are based on the most widely used techniques for this problem—Graph Convolutional Networks (GCNs), Temporal Convolutional Networks (TCNs) and Recurrent Neural Networks (RNNs). Initially, the paper explores and compares different ways to extract the most relevant spatial and temporal characteristics for a sequence of frames describing an action. Based on this comparative analysis, we show how a TCN type unit can be extended to work even on the characteristics extracted from the spatial domain. To validate our approach, we test it against a benchmark often used for human action recognition problems and we show that our solution obtains comparable results to the state-of-the-art, but with a significant increase in the inference speed.

In real-time applications, the aim of speech enhancement (SE) is to achieve optimal performance while ensuring computational efficiency and near-instant outputs. Many deep neural models have achieved optimal performance in terms of speech quality and intelligibility. However, formulating efficient and compact deep neural models for real-time processing on resource-limited devices remains a challenge. This study presents a compact neural model designed in a complex frequency domain for speech enhancement, optimized for resource-limited devices. The proposed model combines convolutional encoder–decoder and recurrent architectures to effectively learn complex mappings from noisy speech for real-time speech enhancement, enabling low-latency causal processing. Recurrent architectures such as Long-Short Term Memory (LSTM), Gated Recurrent Unit (GRU), and Simple Recurrent Unit (SRU), are incorporated as bottlenecks to capture temporal dependencies and improve the performance of SE. By representing the speech in the complex frequency domain, the proposed model processes both magnitude and phase information. Further, this study extends the proposed models and incorporates attention-gate-based skip connections, enabling the models to focus on relevant information and dynamically weigh the important features. The results show that the proposed models outperform the recent benchmark models and obtain better speech quality and intelligibility. The proposed models show less computational load and deliver better results. This study uses the WSJ0 database where clean sentences from WSJ0 are mixed with different background noises to create noisy mixtures. The results show that STOI and PESQ are improved by 21.1% and 1.25 (41.5%) on the WSJ0 database whereas, on the VoiceBank+DEMAND database, STOI and PESQ are improved by 4.1% and 1.24 (38.6%) respectively. The extension of the models shows further improvement in STOI and PESQ in seen and unseen noisy conditions. • Proposed Deep Models with LSTM, GRU, and SRU Model for Speech Enhancement. • To Capture Temporal Sequences, LSTM, GRU, and SRU are applied to extracted Spectral Features. • Attention Gates are added to the Skip Connections to Focus on Important Spectral Features. • The Proposed SE Models are Evaluated on two datasets (WSJ0 and VoiceBank+DEMAND). [ABSTRACT FROM AUTHOR]

Large data computing is a research problem and a major challenge in order to successfully mine, process, and evaluate massive datasets, as they represent a useful source of knowledge across multiple and intersecting domains. This paper explores the impact of Dimensionality Reduction (DR) on estimating the force in robotic-assisted surgery using recurrent and recurrent convolutional neural networks. This work collects an extensive dataset from three ex vivo porcine samples and one ex vivo artificial skin, as well as from various sensors, surgical tools, and manipulators, to research the impact of dimensionality reduction. Three neural networks were considered to analyze and validate the results of this work: Recurrent Neural Networks (RNN-LSTM), Recurrent Convolutional Neural Networks (RCNN), and Modified Inception ResNet V2. The statistical analysis of the estimated force quality shows a 17.33% improvement in Ince. ResNet V2 networks, a 10.08% improvement in CNN+LSTM networks, and a 3.88% improvement in RNN-LSTM networks with and without dimensionality reduction in the average of all the force components in all three datasets. The analysis also shows a reduction in training time of 8.21% in LSTM-RNN, 14.91% in CNN-LSTM, and 21.01% in Ince. ResNet V2 with and without DR. Additionally, networks with DR outperform those without DR in terms of execution time per step and force prediction time, resulting in notable reductions in each aspect. Sensitivity analysis reveals that Torque, Position, Deformation, Stiffness, Tool diameter, Rotation, and Orientation features have significant impacts on the predicted force after DR. It was also observed that the predicted force quality was superior when performing feature selection and dimensionality reduction on collected features from tool, manipulator, tissue, and vision data when processed simultaneously in all three architectures. The findings have significant implications for various fields and applications beyond the specific domain of surgical robotics. • Impact of dimensionality reduction explored in RAMIS using Recurrent neural networks. • PCC and PCA+GDA based features extraction method increases force estimation accuracy. • ResNet V2 stands out best model due to data handling and identity mapping capability. • RCNNs learning and DR improve the estimation accuracy, execution and prediction time. • Sensitivity analysis identifies key features and their impact on force estimation. [ABSTRACT FROM AUTHOR]

Objective: We investigate the design of deep recurrent neural networks for detecting sleep stages from single channel EEG signals recorded at home by non-expert users. We report the effect of data set size, architecture choices, regularization, and personalization on the classification performance.Methods: We evaluated 58 different architectures and training configurations using three-fold cross validation.Results: A network consisting of convolutional (CONV) layers and long short term memory (LSTM) layers can achieve an agreement with a human annotator of Cohen's Kappa of ~0.73 using a training data set of 19 subjects. Regularization and personalization do not lead to a performance gain.Conclusion: The optimal neural network architecture achieves a performance that is very close to the previously reported human inter-expert agreement of Kappa 0.75.Significance: We give the first detailed account of CONV/LSTM network design process for EEG sleep staging in single channel home based setting.

A goal of systems neuroscience is to discover the circuit mechanisms underlying brain function. Despite experimental advances that enable circuit-wide neural recording, the problem remains open in part because solving the ‘inverse problem’ of inferring circuity and mechanism by merely observing activity is hard. In the grid cell system, we show through modeling that a technique based on global circuit perturbation and examination of a novel theoretical object called the distribution of relative phase shifts (DRPS) could reveal the mechanisms of a cortical circuit at unprecedented detail using extremely sparse neural recordings. We establish feasibility, showing that the method can discriminate between recurrent versus feedforward mechanisms and amongst various recurrent mechanisms using recordings from a handful of cells. The proposed strategy demonstrates that sparse recording coupled with simple perturbation can reveal more about circuit mechanism than can full knowledge of network activity or the synaptic connectivity matrix.

Classification of very high-resolution (VHR) satellite images has three major challenges: 1) inherent low intraclass and high interclass spectral similarities; 2) mismatching resolution of available bands; and 3) the need to regularize noisy classification maps. Conventional methods have addressed these challenges by adopting separate stages of image fusion, feature extraction, and postclassification map regularization. These processing stages, however, are not jointly optimizing the classification task at hand. In this paper, we propose a single-stage framework embedding the processing stages in a recurrent multiresolution convolutional network trained in an end-to-end manner. The feedforward version of the network, called FuseNet, aims to match the resolution of the panchromatic and multispectral bands in a VHR image using convolutional layers with corresponding downsampling and upsampling operations. Contextual label information is incorporated into FuseNet by means of a recurrent version called ReuseNet. We compared FuseNet and ReuseNet against the use of separate processing steps for both image fusions, e.g., pansharpening and resampling through interpolation and map regularization such as conditional random fields. We carried out our experiments on a land-cover classification task using a Worldview-03 image of Quezon City, Philippines, and the International Society for Photogrammetry and Remote Sensing 2-D semantic labeling benchmark data set of Vaihingen, Germany. FuseNet and ReuseNet surpass the baseline approaches in both the quantitative and qualitative results. [ABSTRACT FROM AUTHOR]

Objective: We investigate the design of deep recurrent neural networks for detecting sleep stages from single channel EEG signals recorded at home by non-expert users. We report the effect of data set size, architecture choices, regularization, and personalization on the classification performance. Methods: We evaluated 58 different architectures and training configurations using three-fold cross validation. Results: A network consisting of convolutional (CONV) layers and long short term memory (LSTM) layers can achieve an agreement with a human annotator of Cohen's Kappa of ~0.73 using a training data set of 19 subjects. Regularization and personalization do not lead to a performance gain. Conclusion: The optimal neural network architecture achieves a performance that is very close to the previously reported human inter-expert agreement of Kappa 0.75. Significance: We give the first detailed account of CONV/LSTM network design process for EEG sleep staging in single channel home based setting. [ABSTRACT FROM AUTHOR]

Neuronal systems have a high propensity to engage in oscillatory activity because both the properties of individual neurons and canonical circuit motifs favour rhythmic activity. In addition, coupled oscillators can engage in a large variety of dynamical regimes, ranging from synchronization with different phase offsets to chaotic behaviour. Which regime prevails depends on differences between preferred oscillation frequencies, coupling strength and coupling delays. The ability of delay coupled oscillator networks to generate a rich repertoire of temporally structured activation sequences is exploited by central pattern generator networks for the control of movements. However, it is less clear whether temporal patterning of neuronal discharges also plays a role in cognitive processes. Here, it will be argued that the temporal patterning of neuronal discharges emerging from delay coupled oscillator networks plays a pivotal role in all instances in which selective relations have to be established between the responses of distributed assemblies of neurons. Examples are the dynamic formation of functional networks, the selective routing of activity in densely interconnected networks, the attention‐dependent selection of sensory signals, the fast and context‐dependent binding of responses for further joint processing in pattern recognition and the formation of associations by learning. Special consideration is given to arguments that challenge a functional role of oscillations and synchrony in cognition because of the volatile nature of these phenomena and recent evidence will be reviewed suggesting that this volatility is functionally advantageous. This review discusses the functional role of neuronal oscillations and synchrony. The cerebral cortex is considered as a delay coupled recurrent network whose nodes consist of feature selective oscillatory microcircuits. It is concluded that the emerging dynamics endow neuronal responses with the temporal structure required for the definition of semantic relations in the context of feature binding, the formation of functional networks and the establishment of engrams. [ABSTRACT FROM AUTHOR]

In the domain of sequence modelling, Recurrent Neural Networks (RNN) have been capable of achieving impressive results in a variety of application areas including visual question answering, part-of-speech tagging and machine translation. However this success in modelling short term dependencies has not successfully transitioned to application areas such as trajectory prediction, which require capturing both short term and long term relationships. In this paper, we propose a Tree Memory Network (TMN) for jointly modelling both long term relationships between multiple sequences and short term relationships within a sequence, in sequence-to-sequence mapping problems. The proposed network architecture is composed of an input module, controller and a memory module. In contrast to related literature which models the memory as a sequence of historical states, we model the memory as a recursive tree structure. This structure more effectively captures temporal dependencies across both short and long term time periods through its hierarchical structure. We demonstrate the effectiveness and flexibility of the proposed TMN in two practical problems: aircraft trajectory modelling and pedestrian trajectory modelling in a surveillance setting. In both cases the proposed approach outperforms the current state-of-the-art. Furthermore, we perform an in depth analysis on the evolution of the memory module content over time and provide visual evidence on how the proposed TMN is able to map both short and long term relationships efficiently via a hierarchical structure. [ABSTRACT FROM AUTHOR]

Neural-network-based image and video captioning can be substantially improved by utilizing architectures that make use of special features from the scene context, objects, and locations. A novel discriminatively trained evaluator network for choosing the best caption among those generated by an ensemble of caption generator networks further improves accuracy. [ABSTRACT FROM AUTHOR]

Exo-atmospheric infrared (IR) point target discrimination is an important research topic of space surveillance systems. It is difficult to describe the characteristic information of the shape and micro-motion states of the targets and to discriminate different targets effectively by the characteristic information. This paper has constructed the infrared signature model of spatial point targets and obtained the infrared radiation intensity sequences dataset of different types of targets. This paper aims to design an algorithm for the classification problem of infrared radiation intensity sequences of spatial point targets. Recurrent neural networks (RNNs) are widely used in time series classification tasks, but face several problems such as gradient vanishing and explosion, etc. In view of shortcomings of RNNs, this paper proposes an independent random recurrent neural network (IRRNN) model, which combines independent structure RNNs with randomly weighted RNNs. Without increasing the training complexity of network learning, our model solves the problem of gradient vanishing and explosion, improves the ability to process long sequences, and enhances the comprehensive classification performance of the algorithm effectively. Experiments show that the IRRNN algorithm performs well in classification tasks and is robust to noise.

We examine a family of random firing-rate neural networks in which we enforce the neurobiological constraint of Dale's Law-each neuron makes either excitatory or inhibitory connections onto its post-synaptic targets. We find that this constrained system may be described as a perturbation from a system with nontrivial symmetries. We analyze the symmetric system using the tools of equivariant bifurcation theory and demonstrate that the symmetry-implied structures remain evident in the perturbed system. In comparison, spectral characteristics of the network coupling matrix are relatively uninformative about the behavior of the constrained system. [ABSTRACT FROM AUTHOR]

In this paper, a recurrent neural network coupled with Kalman filter is proposed to identify dynamic terms of robotic manipulator. By cooperating some inherent characteristics of robot, this network has the capability to individually identify nonlinear terms using Weighted Augmentation Error (WAE). To present the infrastructure of architecture, an adaptive scheme based on the conventional Back Propagation (BP) is firstly driven using the Gradient Descent (GD) method. Additionally, a stable adaptive updating rule is extracted from the discrete time Lyapunov candidate as an approach for the general nonlinear system identification. Then, this approach is applied to the predefined network. To experimentally validate the computational efficiency and control applicability of the proposed method, Adaptive Neural Network Based Inverse Dynamic Control (ANN-Based-IDC) is employed on a laboratory-scaled twin-rotor CE-150 helicopter. This experiment illustrates enhancement of steady-state performance from 2-to-3 times more in compared with simple PID. Moreover, disturbance rejection and robustness tests admit capability of the method for online dynamic identification in the presence of output and dynamic perturbation. [ABSTRACT FROM AUTHOR]