Search

Your search keyword '"Li, Siyuan"' showing total 91 results
Start Over Author "Li, Siyuan" Publisher ieee
91 results on '"Li, Siyuan"'

16. Digital Twin and Artificial Intelligence-Empowered Panoramic Video Streaming: Reducing Transmission Latency in the Extended Reality-Assisted Vehicular Metaverse.

Author

Li, Siyuan, Lin, Xi, Wu, Jun, Zhang, Wei, and Li, Jianhua

Abstract

The vehicular metaverse is expected to provide a widely connected virtual Internet of Vehicles (IoV), where extended reality (XR) is one of the critical infrastructures. However, the combination of XR and automated vehicle (AV) networks brings several significant challenges, e.g., low-latency XR panoramic video transmission, high bandwidth, and the high mobility of vehicles. This article introduces digital twin (DT) and artificial intelligence (AI)-empowered panoramic video streaming for XR-assisted connected AVs to reduce transmission latency and intelligently respond to user requirements. Specifically, we propose a DT-enabled distributed XR service management framework to provide low-latency and smooth XR services across different domains in the vehicular metaverse. In addition, we present a case study on XR streaming-based virtualized resource allocation and propose a novel deep reinforcement learning (DRL)-based method to minimize transmission latency. Quantitative experimental results demonstrate that the positive role of AI in connected AV networks can be enhanced by DTs. Finally, open issues and potential research directions for the XR-assisted vehicular metaverse are discussed. [ABSTRACT FROM AUTHOR]

Published
2023
Full Text
View/download PDF

32. UMT: Unified Multi-modal Transformers for Joint Video Moment Retrieval and Highlight Detection

Author

Liu, Ye, Li, Siyuan, Wu, Yang, Chen, Chang Wen, Shan, Ying, and Qie, Xiaohu

Subjects
FOS: Computer and information sciences, Computer Vision and Pattern Recognition (cs.CV), Computer Science - Computer Vision and Pattern Recognition
Abstract

Finding relevant moments and highlights in videos according to natural language queries is a natural and highly valuable common need in the current video content explosion era. Nevertheless, jointly conducting moment retrieval and highlight detection is an emerging research topic, even though its component problems and some related tasks have already been studied for a while. In this paper, we present the first unified framework, named Unified Multi-modal Transformers (UMT), capable of realizing such joint optimization while can also be easily degenerated for solving individual problems. As far as we are aware, this is the first scheme to integrate multi-modal (visual-audio) learning for either joint optimization or the individual moment retrieval task, and tackles moment retrieval as a keypoint detection problem using a novel query generator and query decoder. Extensive comparisons with existing methods and ablation studies on QVHighlights, Charades-STA, YouTube Highlights, and TVSum datasets demonstrate the effectiveness, superiority, and flexibility of the proposed method under various settings. Source code and pre-trained models are available at https://github.com/TencentARC/UMT., Comment: Accepted to Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2022)

Published
2022

37. Generalized Clustering and Multi-Manifold Learning with Geometric Structure Preservation

Author

Wu, Lirong, Liu, Zicheng, Zang, Zelin, Xia, Jun, Li, Siyuan, and Li, Stan. Z

Subjects
FOS: Computer and information sciences, Computer Science - Machine Learning, ComputingMethodologies_PATTERNRECOGNITION, Artificial Intelligence (cs.AI), Computer Science - Artificial Intelligence, Statistics - Machine Learning, Machine Learning (stat.ML), Machine Learning (cs.LG)
Abstract

Though manifold-based clustering has become a popular research topic, we observe that one important factor has been omitted by these works, namely that the defined clustering loss may corrupt the local and global structure of the latent space. In this paper, we propose a novel Generalized Clustering and Multi-manifold Learning (GCML) framework with geometric structure preservation for generalized data, i.e., not limited to 2-D image data and has a wide range of applications in speech, text, and biology domains. In the proposed framework, manifold clustering is done in the latent space guided by a clustering loss. To overcome the problem that the clustering-oriented loss may deteriorate the geometric structure of the latent space, an isometric loss is proposed for preserving intra-manifold structure locally and a ranking loss for inter-manifold structure globally. Extensive experimental results have shown that GCML exhibits superior performance to counterparts in terms of qualitative visualizations and quantitative metrics, which demonstrates the effectiveness of preserving geometric structure.

Published
2022

48. A Shared Representation for Photorealistic Driving Simulators.

Author

Saadatnejad, Saeed, Li, Siyuan, Mordan, Taylor, and Alahi, Alexandre

Abstract

A powerful simulator highly decreases the need for real-world tests when training and evaluating autonomous vehicles. Data-driven simulators flourished with the recent advancement of conditional Generative Adversarial Networks (cGANs), providing high-fidelity images. The main challenge is synthesizing photorealistic images while following given constraints. In this work, we propose to improve the quality of generated images by rethinking the discriminator architecture. The focus is on the class of problems where images are generated given semantic inputs, such as scene segmentation maps or human body poses. We build on successful cGAN models to propose a new semantically-aware discriminator that better guides the generator. We aim to learn a shared latent representation that encodes enough information to jointly do semantic segmentation, content reconstruction, along with a coarse-to-fine grained adversarial reasoning. The achieved improvements are generic and simple enough to be applied to any architecture of conditional image synthesis. We demonstrate the strength of our method on the scene, building, and human synthesis tasks across three different datasets. The code is available https://github.com/vita-epfl/SemDisc. [ABSTRACT FROM AUTHOR]

Published
2022
Full Text
View/download PDF

50. Multi-objective Optimal Control Based on Practical Security Region of Regional Integrated Energy System

Author

Hongjie Jia, Yanbin Zhu, Dan Wang, Li Siyuan, and Liu Liu

Subjects
Set (abstract data type), Mathematical optimization, Computer science, 020209 energy, 0206 medical engineering, Control (management), 0202 electrical engineering, electronic engineering, information engineering, Energy balance, 02 engineering and technology, Optimal control, 020601 biomedical engineering, Integrated energy system
Abstract

In view of the requirement of regional integrated energy system (RIES) for safe and efficient operation, scholars researched how to adjust RIES to a balanced status of security and efficiency on the premise of satisfying the N-l guideline has been researched. However, the rapidity of the control strategy, which represents the adjustment amount, and economy of it hasn’t been considered. Based on the practical regional integrated energy system security region (RIESSR), a new approach to optimal control problem of RIES is proposed to minimize control cost and adjustment amount as well as maximizing security degree and efficiency degree. It can adjust RIES from a secure status satisfying the N-l guideline to a balanced status of security and efficiency with economical and rapid control strategy. Meanwhile, the constraints include security inequality constraints of hyperplane form and multi-energy flow energy balance equations. Finally, a specific case is set to verify effectiveness of the proposed optimal control method.

Published
2020

Catalog

Books, media, physical & digital resources
See catalog results