Back to Search
Start Over
Relation classification via BERT with piecewise convolution and focal loss
- Source :
- PLoS ONE, Vol 16, Iss 9, p e0257092 (2021), PLoS ONE
- Publication Year :
- 2021
- Publisher :
- Public Library of Science (PLoS), 2021.
-
Abstract
- Recent relation extraction models’ architecture are evolved from the shallow neural networks to natural language model, such as convolutional neural networks or recurrent neural networks to Bert. However, these methods did not consider the semantic information in the sequence or the distance dependence problem, the internal semantic information may contain the useful knowledge which can help relation classification. Focus on these problems, this paper proposed a BERT-based relation classification method. Compare with the existing Bert-based architecture, the proposed model can obtain the internal semantic information between entity pair and solve the distance semantic dependence better. The pre-trained BERT model after fine tuning is used in this paper to abstract the semantic representation of sequence, then adopt the piecewise convolution to obtain semantic information which influence the extraction results. Compare with the existing methods, the proposed method can achieve a better accuracy on relational extraction task because of the internal semantic information extracted in the sequence. While, the generalization ability is still a problem that cannot be ignored, and the numbers of the relationships are difference between different categories. In this paper, the focal loss function is adopted to solve this problem by assigning a heavy weight to less number or hard classify categories. Finally, comparing with the existing methods, the F1 metric of the proposed method can reach a superior result 89.95% on the SemEval-2010 Task 8 dataset.
- Subjects :
- Computer and Information Sciences
Neural Networks
Computer science
Science
Social Sciences
Semantics
Research and Analysis Methods
Convolutional neural network
Machine Learning
Deep Learning
Mathematical and Statistical Techniques
Artificial Intelligence
Psychology
Attention
Syntax
Recurrent Neural Networks
Natural Language Processing
Grammar
Multidisciplinary
Artificial neural network
business.industry
Deep learning
Cognitive Psychology
Biology and Life Sciences
Pattern recognition
Linguistics
Models, Theoretical
Relationship extraction
Convolution
Recurrent neural network
Databases as Topic
Metric (mathematics)
Piecewise
Cognitive Science
Medicine
Artificial intelligence
Neural Networks, Computer
business
Mathematical Functions
Algorithms
Research Article
Neuroscience
Subjects
Details
- Language :
- English
- ISSN :
- 19326203
- Volume :
- 16
- Issue :
- 9
- Database :
- OpenAIRE
- Journal :
- PLoS ONE
- Accession number :
- edsair.doi.dedup.....b0a853dd7151e8328bb34ebf371b0506