Back to Search
Start Over
Graph-Based Visual-Semantic Entanglement Network for Zero-Shot Image Recognition
- Source :
- IEEE Transactions on Multimedia. 24:2473-2487
- Publication Year :
- 2022
- Publisher :
- Institute of Electrical and Electronics Engineers (IEEE), 2022.
-
Abstract
- Zero-shot learning uses semantic attributes to connect the search space of unseen objects. In recent years, although the deep convolutional network brings powerful visual modeling capabilities to the ZSL task, its visual features have severe pattern inertia and lack of representation of semantic relationships, which leads to severe bias and ambiguity. In response to this, we propose the Graph-based Visual-Semantic Entanglement Network to conduct graph modeling of visual features, which is mapped to semantic attributes by using a knowledge graph, it contains several novel designs: 1. it establishes a multi-path entangled network with the convolutional neural network (CNN) and the graph convolutional network (GCN), which input the visual features from CNN to GCN to model the implicit semantic relations, then GCN feedback the graph modeled information to CNN features; 2. it uses attribute word vectors as the target for the graph semantic modeling of GCN, which forms a self-consistent regression for graph modeling and supervise GCN to learn more personalized attribute relations; 3. it fuses and supplements the hierarchical visual-semantic features refined by graph modeling into visual embedding. Our method outperforms state-of-the-art approaches on multiple representative ZSL datasets: AwA2, CUB, and SUN by promoting the semantic linkage modelling of visual features.<br />15 pages, 11 figures, on IEEE Transactions on Multimedia
- Subjects :
- FOS: Computer and information sciences
Computer Science - Machine Learning
Computer science
Computer Vision and Pattern Recognition (cs.CV)
media_common.quotation_subject
Computer Science - Computer Vision and Pattern Recognition
Space (commercial competition)
Convolutional neural network
Machine Learning (cs.LG)
FOS: Electrical engineering, electronic engineering, information engineering
Media Technology
Electrical and Electronic Engineering
Representation (mathematics)
media_common
business.industry
Image and Video Processing (eess.IV)
Pattern recognition
Ambiguity
Electrical Engineering and Systems Science - Image and Video Processing
Computer Science Applications
Signal Processing
Graph (abstract data type)
Embedding
Visual modeling
Artificial intelligence
business
Word (computer architecture)
Subjects
Details
- ISSN :
- 19410077 and 15209210
- Volume :
- 24
- Database :
- OpenAIRE
- Journal :
- IEEE Transactions on Multimedia
- Accession number :
- edsair.doi.dedup.....df9ab445a3eb5c74c161277a3a54b386