Back to Search
Start Over
Cross-modal recipe retrieval with stacked attention model
- Source :
- Multimedia Tools and Applications. 77:29457-29473
- Publication Year :
- 2018
- Publisher :
- Springer Science and Business Media LLC, 2018.
-
Abstract
- Taking a picture of delicious food and sharing it in social media has been a popular trend. The ability to recommend recipes along will benefit users who want to cook a particular dish, and the feature is yet to be available. The challenge of recipe retrieval, nevertheless, comes from two aspects. First, the current technology in food recognition can only scale up to few hundreds of categories, which are yet to be practical for recognizing tens of thousands of food categories. Second, even one food category can have variants of recipes that differ in ingredient composition. Finding the best-match recipe requires knowledge of ingredients, which is a fine-grained recognition problem. In this paper, we consider the problem from the viewpoint of cross-modality analysis. Given a large number of image and recipe pairs acquired from the Internet, a joint space is learnt to locally capture the ingredient correspondence between images and recipes. As learning happens at the regional level for image and ingredient level for recipe, the model has the ability to generalize recognition to unseen food categories. Furthermore, the embedded multi-modal ingredient feature sheds light on the retrieval of best-match recipes. On an in-house dataset, our model can double the retrieval performance of DeViSE, a popular cross-modality model but not considering region information during learning.
- Subjects :
- Information retrieval
Computer Networks and Communications
business.industry
Computer science
Recipe
02 engineering and technology
010501 environmental sciences
Space (commercial competition)
01 natural sciences
Image (mathematics)
Ingredient
Modal
Hardware and Architecture
0202 electrical engineering, electronic engineering, information engineering
Media Technology
Feature (machine learning)
020201 artificial intelligence & image processing
The Internet
Social media
business
Software
0105 earth and related environmental sciences
Subjects
Details
- ISSN :
- 15737721 and 13807501
- Volume :
- 77
- Database :
- OpenAIRE
- Journal :
- Multimedia Tools and Applications
- Accession number :
- edsair.doi...........16769a57594c23006cfada7e60742924
- Full Text :
- https://doi.org/10.1007/s11042-018-5964-y