Multiple instance relation graph reasoning for cross-modal hash retrieval.

The similarity calculation is too simple in most cross-modal hash retrieval methods, which do not consider the impact of the relations between instances. To solve this problem, this paper proposes a reasoning method based on multiple instance relation graphs. By constructing similarity matrices, we establish global and local instance relation graphs, which fully exploit fine-grained relations between instances. First, we perform relation reasoning based on the relation graphs of the image and text modalities; then, we map the relations within the two modalities into the instance graphs; finally, we perform relation reasoning based on the instance graphs. Furthermore, to accommodate the features of both the image and text modalities, we employ a step-by-step training strategy to train the proposed neural network model. According to the results of experiments on the MIRFlickr and NUS-WIDE datasets, our method has apparent advantages in terms of m A P and has a good effect on the topK-precision curve. This shows that our method realizes the in-depth mining of instance relations, which can improve the retrieval performance significantly. • We compute complex relations between instances through two evaluation criteria. • We construct three kinds of relation graphs to mine fine-grained relations. • We carry out three kinds of reasoning based on the constructed relation graphs. • We propose a new step-by-step strategy for training the network framework. [ABSTRACT FROM AUTHOR]