Search

Recurrent Neural Networks (RNNs) are classical models for processing sequential data, demonstrating excellent performance in tasks such as natural language processing and time series prediction. However, during the training of RNNs, the issues of vanishing and exploding gradients often arise, significantly impacting the model's performance and efficiency. In this paper, we investigate why RNNs are more prone to gradient problems compared to other common sequential networks. To address this issue and enhance network performance, we propose a method for gradient normalization of network weights. This method suppresses the occurrence of gradient problems by altering the statistical properties of RNN weights, thereby improving training effectiveness. Additionally, we analyze the impact of weight gradient normalization on the probability-distribution characteristics of model weights and validate the sensitivity of this method to hyperparameters such as learning rate. The experimental results demonstrate that gradient normalization enhances the stability of model training and reduces the frequency of gradient issues. On the Penn Treebank dataset, this method achieves a perplexity level of 110.89, representing an 11.48% improvement over conventional gradient descent methods. For prediction lengths of 24 and 96 on the ETTm1 dataset, Mean Absolute Error (MAE) values of 0.778 and 0.592 are attained, respectively, resulting in 3.00% and 6.77% improvement over conventional gradient descent methods. Moreover, selected subsets of the UCR dataset show an increase in accuracy ranging from 0.4% to 6.0%. The gradient normalization method enhances the ability of RNNs to learn from sequential and causal data, thereby holding significant implications for optimizing the training effectiveness of RNN-based models. [ABSTRACT FROM AUTHOR]

This study investigates the inhibitory effect and potential mechanism of ligustrazine combined with paeonol on hepatic fibrosis, as to provide a new therapeutic strategy for clinical hepatic fibrosis. The degree of liver injury collagen deposition and inflammation was assessed by hematoxylin and eosin staining, Masson, Sirius red staining and biochemically serum analysis. ATP and ROS levels in each group were detected by chemical fluorescence method. The apoptotic rate was measured by Tunel assay. Mito-Tacker fluorescence staining and mitochondrial DNA copy number were measured to observe the effect of ligustrazine or/and paeonol on mitochondrial function of hepatic stellate cell (HSC). The expression of relevant proteins and genes were evaluated by using immunofluorescence RT-PCR and western blot. Ligustrazine or/and paeonol significantly improve the pathological changes in liver tissue induced by CCl 4 , however, they reduced the levels of liver and fibrosis markers in tissue and serum. ROS, NOX1 and NOX2 were significantly increased and GSH was decreased in HSC, with the intervention of Ligustrazine or/and paeonol. We further found that Ligustrazine or/and paeonol can effectively inhibit liver inflammation in vivo. The expression of TNF-α, IL-6 and IL-8 was upregulated in HSC. Moreover, Ligustrazine or/and paeonol promotes apoptosis and inhibit proliferation of HSC. Additionally, the inhibiting effects of the drug on collagen deposition was due to the interference with the expression of signaling pathway related proteins and genes such as, MMPS, TGF-β, PDGF and BMP-2 in HSC. Mitochondrial activity of HSC was inhibited by Ligustrazine or/and paeonol. The inhibitory effects of ligustrazine or/and Paeonol on mitochondrial function is partially balanced by mitochondrial protective agent SS-31. Ligustrazine combined with paeonol exerts significant anti-hepatic fibrosis effect in vivo and in vitro. This may due to the disruption of HSC mitochondrial function, thereby induced promoting oxidative stress, apoptosis, inflammation and inhibiting the formation and deposition of extracellular matrix., Competing Interests: None., (AJTR Copyright © 2020.)