Search

Your search keyword '"Lin, Xudong"' showing total 41 results
Start Over Author "Lin, Xudong" Publisher ieee
41 results on '"Lin, Xudong"'

4. All in One: Exploring Unified Video-Language Pre-Training

Author

Wang, Jinpeng, primary, Ge, Yixiao, additional, Yan, Rui, additional, Ge, Yuying, additional, Lin, Kevin Qinghong, additional, Tsutsui, Satoshi, additional, Lin, Xudong, additional, Cai, Guanyu, additional, Wu, Jianping, additional, Shan, Ying, additional, Qie, Xiaohu, additional, and Shou, Mike Zheng, additional

Published
2023
Full Text
View/download PDF

8. CLIP-Event: Connecting Text and Images with Event Structures

Author

Li, Manling, Xu, Ruochen, Wang, Shuohang, Zhou, Luowei, Lin, Xudong, Zhu, Chenguang, Zeng, Michael, Ji, Heng, and Chang, Shih-Fu

Subjects
FOS: Computer and information sciences, Artificial Intelligence (cs.AI), Computer Science - Artificial Intelligence, Computer Vision and Pattern Recognition (cs.CV), Computer Science - Computer Vision and Pattern Recognition
Abstract

Vision-language (V+L) pretraining models have achieved great success in supporting multimedia applications by understanding the alignments between images and text. While existing vision-language pretraining models primarily focus on understanding objects in images or entities in text, they often ignore the alignment at the level of events and their argument structures. In this work, we propose a contrastive learning framework to enforce vision-language pretraining models to comprehend events and associated argument (participant) roles. To achieve this, we take advantage of text information extraction technologies to obtain event structural knowledge, and utilize multiple prompt functions to contrast difficult negative descriptions by manipulating event structures. We also design an event graph alignment loss based on optimal transport to capture event argument structures. In addition, we collect a large event-rich dataset (106,875 images) for pretraining, which provides a more challenging image retrieval benchmark to assess the understanding of complicated lengthy sentences. Experiments show that our zero-shot CLIP-Event outperforms the state-of-the-art supervised model in argument extraction on Multimedia Event Extraction, achieving more than 5% absolute F-score gain in event extraction, as well as significant improvements on a variety of downstream tasks under zero-shot settings.

Published
2022

9. Object-aware Video-language Pre-training for Retrieval

Author

Wang, Alex Jinpeng, Ge, Yixiao, Cai, Guanyu, Yan, Rui, Lin, Xudong, Shan, Ying, Qie, Xiaohu, and Shou, Mike Zheng

Subjects
FOS: Computer and information sciences, Computer Science - Computation and Language, Computer Vision and Pattern Recognition (cs.CV), Computer Science - Computer Vision and Pattern Recognition, Computation and Language (cs.CL)
Abstract

Recently, by introducing large-scale dataset and strong transformer network, video-language pre-training has shown great success especially for retrieval. Yet, existing video-language transformer models do not explicitly fine-grained semantic align. In this work, we present Object-aware Transformers, an object-centric approach that extends video-language transformer to incorporate object representations. The key idea is to leverage the bounding boxes and object tags to guide the training process. We evaluate our model on three standard sub-tasks of video-text matching on four widely used benchmarks. We also provide deep analysis and detailed ablation about the proposed method. We show clear improvement in performance across all tasks and datasets considered, demonstrating the value of a model that incorporates object representations into a video-language architecture. The code will be released at \url{https://github.com/FingerRec/OA-Transformer}., CVPR2022; Code: https://github.com/FingerRec/OA-Transformer

Published
2022

22. Experimental Demonstration of Sequential Operation Approach for a Three-Sided Pyramid Wavefront Sensor.

Author

Chen, Lu, Wang, Jianli, Yao, Kainan, Liu, Xinyue, Lin, Xudong, Wang, Liang, and Wang, Minghao

Abstract

We describe a sequential operation approach for a three-sided pyramid wavefront sensor (PWFS) based on a micromirror array with hexagonal geometry. The geometrical optics analysis and numerical simulations show that a linear relationship exists between the signals calculated by the sensing data and the local tilt of the wavefront. We discuss the selection of parameter index that is used in our experiment and build a closed-loop system with this wavefront sensor and a 21-element deformable mirror as a wavefront corrector. The experimental results confirm the feasibility of using a nonmodulated sequential three-sided PWFS in adaptive optics systems. [ABSTRACT FROM PUBLISHER]

Published
2016
Full Text
View/download PDF

Catalog

Books, media, physical & digital resources
See catalog results