Back to Search
Start Over
LDA-Reg: Knowledge Driven Regularization Using External Corpora.
- Source :
- IEEE Transactions on Knowledge & Data Engineering; Dec2022, Vol. 34 Issue 12, p5840-5853, 14p
- Publication Year :
- 2022
-
Abstract
- While recent developments of neural network (NN) models have led to a series of record-breaking achievements in many applications, the lack of sufficiently good datasets remains a problem for some applications. For such a problem, we can however exploit a large number of unstructured text corpora as an external knowledge to complement the training data, and most prevailing neural network solutions employ word embedding methods for such purposes. In this paper, we propose LDA-Reg, a novel knowledge driven regularization framework based on Latent Dirichlet Allocation (LDA) as an alternative to the word embedding methods to adaptively utilize abundant external knowledge and to interpret the NN model. For the joint learning of the parameters, we propose EM-SGD, an effective update method which incorporates Expectation Maximization (EM) and Stochastic Gradient Descent (SGD) to update parameters iteratively. Moreover, we also devise a lazy update and sparse update method for the high-dimensional inputs and sparse inputs respectively. We validate the effectiveness of our regularization framework through an extensive experimental study over real world and standard benchmark datasets. The results show that our proposed framework not only achieves significant improvement over state-of-the-art word embedding methods but also learns interpretable and significant topics for various tasks. [ABSTRACT FROM AUTHOR]
- Subjects :
- CORPORA
NEURAL development
DATA mining
ARTIFICIAL neural networks
Subjects
Details
- Language :
- English
- ISSN :
- 10414347
- Volume :
- 34
- Issue :
- 12
- Database :
- Complementary Index
- Journal :
- IEEE Transactions on Knowledge & Data Engineering
- Publication Type :
- Academic Journal
- Accession number :
- 160692101
- Full Text :
- https://doi.org/10.1109/TKDE.2021.3069861