Search

KG-augmented models usually endow existing models with external knowledge graphs, which achieve promising performance in various knowledge-intensive tasks, such as commonsense reasoning. Existing methods mainly first exploited heuristic ways for retrieving the relevant knowledge subgraphs according to the input, and then utilized some effective encoders, such as GNNs, to encode the symbolic knowledge into the neural reasoning networks. However, whether the whole retrieved knowledge subgraphs are really relevant or useful for the reasoning process was seldom considered. Actually, according to our observations and analysis, most retrieved knowledge is noisy and useless to the reasoning models, which would hurt the final performance. To remedy this, this paper proposes information bottleneck based knowledge selection (IBKS), which is able to select useful knowledge from the retrieved knowledge subgraph. Expectedly, the selected knowledge could better improve the commonsense reasoning ability of the model. Moreover, IBKS is model-agnostic and could be plugged into any existing KG-augmented model. Extensive experimental results show that IBKS could effectively improve commonsense reasoning performance. [ABSTRACT FROM AUTHOR]

During the last two decades, evolving fuzzy systems (EFSs) have attracted more and more attention owing to their capacity to self-adapt both system structures and parameters dynamically. Conventional EFSs optimize the consequent parameters using the recursive least square (RLS) alg orithm based on mean square error (MSE) criterion. It only works well in Gaussian noise or noise-free situations. In this paper, we further propose an alternative approach to update consequent parameters based on the novel least mean dual p-power (LMDP) error criterion. By contrast with the MSE criterion, the LMDP criterion has a stronger noise rejection ability on non-Gaussian noise environments. Moreover, the LMDP criterion extends the range of p value in conventional LMP criterion from positive integer to floating point for more fine-grained optimum solutions. Details in this work, the recursive solution of LMDP criterion is respectively derived from global learning and local learning to update consequent parameter and the theoretical stability analyses for both learning cases are proven in that the stability of our considered EFS is guaranteed. The experiments on various benchmark datasets and a real-world typhoon path prediction validate that the proposed method is more accurate and owns better generalization performance under multiple noise conditions. [ABSTRACT FROM AUTHOR]

Evolving fuzzy systems (EFSs) are a type of adaptive fuzzy rule-based systems which can self-adapt both their structures and parameters simultaneously. However, the existing EFSs suffer from two drawbacks: 1) classical EFSs usually use all input features to model systems, resulting in lengthy fuzzy rules; 2) some redundant information in fuzzy rules may hinder high generalization. To address these two issues, a promising method is proposed in this paper by combining very sparse random projection (VSRP) with a class of EFSs based-on data clouds, called VSRP-AnYa-EFS. The proposed method introduces: 1) a random sparse-Bernoulli (RSB) matrix based-on VSRP is utilized to compress the lengthy antecedent part into a tighter form, triggering a feature-reduction mechanism. By employing VSRP in RSB matrix, some redundant information in fuzzy rules can be filtered; 2) Local learning is used for consequent parameter optimization to suit decoupled behavior of rules after redundant information between rules is deleted. By adopting VSRP and local learning, the proposed VSRP-AnYa-EFS owns a compact structure and fast learning speed. Numerical examples presented in this paper demonstrate that the proposed method can significantly reduce training time from hours to minutes while the accuracy can be improved up to 5%. [ABSTRACT FROM AUTHOR]

• The model proposed integrates the advantages of each of the width model (DDR) and depth model (DTDR). • A time delay operator is inserted between adjacent layers to control the transmission process of feature information. • The short-term memory capacity of the model proposed was significantly improved. • The model proposed has higher forecasting accuracy for long-term memory datasets. The forecasting of nonlinear time series has a wide range of applications both in theoretical research and industrial production, as the fitting of nonlinear systems, the early warning of sunspot activity and the prediction of network traffic peaks in advance. These tasks require models with strong nonlinear mapping capabilities and sufficient short-term memory capacity. Single-node time delay Reservoir (TDR) have been frequently applied on nonlinear time series forecasting tasks. To address the lack of memory capacity of traditional TDR and its extended version (Delay Decoupled Reservoir (DDR), Deep Time-Delay Reservoir (DTDR)), we propose a Depth Asynchronous Time-Delay Reservoir (DATDR) model. Firstly, the model retains the deep network structure of the DTDR to ensure the dynamic properties of the model, and the history states of each layer are from the previous layers instead of the current layer. Secondly, the same signal input method as the DDR is used, i.e., each layer is fed with the original signal, and this data input method ensures that the feature of signal does not decay gradually during the transmission between layers. Finally, a time delay operator is inserted into two adjacent layers, and the memory capacity of the model could be controlled effectively by adjusting the delay time of the time delay operator to accommodate some time series prediction tasks that require long memory capacity. In this paper, eight datasets from three different types are used to validate the proposed model, including NARMA10-30, Mackey-Glass Chaos time series, He ́non map series, Sunspot Sequence, and two real network traffic datasets. Experiments show that proposed model gets significant improvement in memory capacity compared with TDR, DDR, and DTDR, and performs better for some nonlinear time series datasets especially some long-term memory datasets. [ABSTRACT FROM AUTHOR]