8 results on '"Lu, Minlong"'
Search Results
2. Product1M: Towards Weakly Supervised Instance-Level Product Retrieval via Cross-Modal Pretraining
- Author
-
Zhan, Xunlin, Wu, Yangxin, Dong, Xiao, Wei, Yunchao, Lu, Minlong, Zhang, Yichi, Xu, Hang, and Liang, Xiaodan
- Subjects
FOS: Computer and information sciences ,Computer Vision and Pattern Recognition (cs.CV) ,Computer Science - Computer Vision and Pattern Recognition - Abstract
Nowadays, customer's demands for E-commerce are more diversified, which introduces more complications to the product retrieval industry. Previous methods are either subject to single-modal input or perform supervised image-level product retrieval, thus fail to accommodate real-life scenarios where enormous weakly annotated multi-modal data are present. In this paper, we investigate a more realistic setting that aims to perform weakly-supervised multi-modal instance-level product retrieval among fine-grained product categories. To promote the study of this challenging task, we contribute Product1M, one of the largest multi-modal cosmetic datasets for real-world instance-level retrieval. Notably, Product1M contains over 1 million image-caption pairs and consists of two sample types, i.e., single-product and multi-product samples, which encompass a wide variety of cosmetics brands. In addition to the great diversity, Product1M enjoys several appealing characteristics including fine-grained categories, complex combinations, and fuzzy correspondence that well mimic the real-world scenes. Moreover, we propose a novel model named Cross-modal contrAstive Product Transformer for instance-level prodUct REtrieval (CAPTURE), that excels in capturing the potential synergy between multi-modal inputs via a hybrid-stream transformer in a self-supervised manner.CAPTURE generates discriminative instance features via masked multi-modal learning as well as cross-modal contrastive pretraining and it outperforms several SOTA cross-modal baselines. Extensive ablation studies well demonstrate the effectiveness and the generalization capacity of our model. Dataset and codes are available at https: //github.com/zhanxlin/Product1M.
- Published
- 2021
3. Learning Spatiotemporal Attention for Egocentric Action Recognition
- Author
-
Lu, Minlong, primary, Liao, Danping, additional, and Li, Ze-Nian, additional
- Published
- 2019
- Full Text
- View/download PDF
4. Deep Attention Network for Egocentric Action Recognition.
- Author
-
Lu, Minlong, Li, Ze-Nian, Wang, Yueming, and Pan, Gang
- Subjects
- *
HUMAN behavior , *TIME-varying networks , *OBJECT manipulation , *SHORT-term memory , *MACHINE learning , *VIDEOS , *VIDEO coding , *PERFORMANCE technology - Abstract
Recognizing a camera wearer’s actions from videos captured by an egocentric camera is a challenging task. In this paper, we employ a two-stream deep neural network composed of an appearance-based stream and a motion-based stream to recognize egocentric actions. Based on the insight that human action and gaze behavior are highly coordinated in object manipulation tasks, we propose a spatial attention network to predict human gaze in the form of attention map. The attention map helps each of the two streams to focus on the most relevant spatial region of the video frames to predict actions. To better model the temporal structure of the videos, a temporal network is proposed. The temporal network incorporates bi-directional long short-term memory to model the long-range dependencies to recognize egocentric actions. The experimental results demonstrate that our method is able to predict attention maps that are consistent with human attention and achieve competitive action recognition performance with the state-of-the-art methods on the GTEA Gaze and GTEA Gaze+ datasets. [ABSTRACT FROM AUTHOR]
- Published
- 2019
- Full Text
- View/download PDF
5. Generating fluent tubes in video synopsis
- Author
-
Lu, Minlong, primary, Wang, Yueming, additional, and Pan, Gang, additional
- Published
- 2013
- Full Text
- View/download PDF
6. Continuous Depth Map Reconstruction From Light Fields.
- Author
-
Li, Jianqiao, Lu, Minlong, and Li, Ze-Nian
- Subjects
- *
DEPTH maps (Digital image processing) , *CONTINUOUS functions , *IMAGE reconstruction , *LIGHT-field cameras , *LINEAR systems - Abstract
In this paper, we investigate how the recently emerged photography technology—the light field—can benefit depth map estimation, a challenging computer vision problem. A novel framework is proposed to reconstruct continuous depth maps from light field data. Unlike many traditional methods for the stereo matching problem, the proposed method does not need to quantize the depth range. By making use of the structure information amongst the densely sampled views in light field data, we can obtain dense and relatively reliable local estimations. Starting from initial estimations, we go on to propose an optimization method based on solving a sparse linear system iteratively with a conjugate gradient method. Two different affinity matrices for the linear system are employed to balance the efficiency and quality of the optimization. Then, a depth-assisted segmentation method is introduced so that different segments can employ different affinity matrices. Experiment results on both synthetic and real light fields demonstrate that our continuous results are more accurate, efficient, and able to preserve more details compared with discrete approaches. [ABSTRACT FROM AUTHOR]
- Published
- 2015
- Full Text
- View/download PDF
7. Accelerometer-Based Gait Recognition by Sparse Representation of Signature Points With Clusters.
- Author
-
Zhang, Yuting, Pan, Gang, Jia, Kui, Lu, Minlong, Wang, Yueming, and Wu, Zhaohui
- Abstract
Gait, as a promising biometric for recognizing human identities, can be nonintrusively captured as a series of acceleration signals using wearable or portable smart devices. It can be used for access control. Most existing methods on accelerometer-based gait recognition require explicit step-cycle detection, suffering from cycle detection failures and intercycle phase misalignment. We propose a novel algorithm that avoids both the above two problems. It makes use of a type of salient points termed signature points (SPs), and has three components: 1) a multiscale SP extraction method, including the localization and SP descriptors; 2) a sparse representation scheme for encoding newly emerged SPs with known ones in terms of their descriptors, where the phase propinquity of the SPs in a cluster is leveraged to ensure the physical meaningfulness of the codes; and 3) a classifier for the sparse-code collections associated with the SPs of a series. Experimental results on our publicly available dataset of 175 subjects showed that our algorithm outperformed existing methods, even if the step cycles were perfectly detected for them. When the accelerometers at five different body locations were used together, it achieved the rank-1 accuracy of 95.8% for identification, and the equal error rate of 2.2% for verification. [ABSTRACT FROM PUBLISHER]
- Published
- 2015
- Full Text
- View/download PDF
8. Visual Cue-Guided Rat Cyborg for Automatic Navigation [Research Frontier].
- Author
-
Wang, Yueming, Lu, Minlong, Wu, Zhaohui, Tian, Liwen, Xu, Kedi, Zheng, Xiaoxiang, and Pan, Gang
- Abstract
A rat robot is a type of animal robots, where an animal is connected to a machine system via a brain-computer interface. Electrical stimuli can be generated by the machine system and delivered to the animal's brain to control its behavior. The sensory capacity and flexible motion ability of rat robots highlight their potential advantages over mechanical robots. However, most existing rat robots require that a human observes the environmental layout to guide navigation, which limits the applications of rat robots. This work incorporates object detection algorithms to a rat robot system to enable it to find 'human-interesting' objects, and then use these cues to guide its behaviors to perform automatic navigation. A miniature camera is mounted on the rat's back to capture the scene in front of the rat. The video is transferred via a wireless module to a computer and we develop some object detection/identification algorithms to allow objects of interest to be found. Next, we make the rat robot perform a specific motion automatically in response to a detected object, such as turning left. A single stimulus does not allow the rat to perform a motion successfully. Inspired by the fact that humans usually give a series of stimuli to a rat robot, we develop a closed-loop model that issues a stimulus sequence automatically according to the state of the rat and the objects in front of it until the rat completes the motion successfully. Thus, the rat robot, which we refer to as a rat cyborg, is able to move according to the detected objects without the need for manual operations. The object detection methods and the closed-loop stimulation model are evaluated in experiments, which demonstrate that our rat cyborg can accomplish human-specified navigation automatically. [ABSTRACT FROM PUBLISHER]
- Published
- 2015
- Full Text
- View/download PDF
Catalog
Discovery Service for Jio Institute Digital Library
For full access to our library's resources, please sign in.