Search

Your search keyword '"Liu, Shuguang"' showing total 2,278 results
Start Over Author "Liu, Shuguang"
2,278 results on '"Liu, Shuguang"'

3. SuperBench: Improving Cloud AI Infrastructure Reliability with Proactive Validation

Author

Xiong, Yifan, Jiang, Yuting, Yang, Ziyue, Qu, Lei, Zhao, Guoshuai, Liu, Shuguang, Zhong, Dong, Pinzur, Boris, Zhang, Jie, Wang, Yang, Jose, Jithin, Pourreza, Hossein, Baxter, Jeff, Datta, Kushal, Ram, Prabhat, Melton, Luke, Chau, Joe, Cheng, Peng, Xiong, Yongqiang, and Zhou, Lidong

Subjects
Computer Science - Distributed, Parallel, and Cluster Computing
Abstract

Reliability in cloud AI infrastructure is crucial for cloud service providers, prompting the widespread use of hardware redundancies. However, these redundancies can inadvertently lead to hidden degradation, so called "gray failure", for AI workloads, significantly affecting end-to-end performance and concealing performance issues, which complicates root cause analysis for failures and regressions. We introduce SuperBench, a proactive validation system for AI infrastructure that mitigates hidden degradation caused by hardware redundancies and enhances overall reliability. SuperBench features a comprehensive benchmark suite, capable of evaluating individual hardware components and representing most real AI workloads. It comprises a Validator which learns benchmark criteria to clearly pinpoint defective components. Additionally, SuperBench incorporates a Selector to balance validation time and issue-related penalties, enabling optimal timing for validation execution with a tailored subset of benchmarks. Through testbed evaluation and simulation, we demonstrate that SuperBench can increase the mean time between incidents by up to 22.61x. SuperBench has been successfully deployed in Azure production, validating hundreds of thousands of GPUs over the last two years., Comment: USENIX ATC '24

Published
2024

5. FP8-LM: Training FP8 Large Language Models

Author

Peng, Houwen, Wu, Kan, Wei, Yixuan, Zhao, Guoshuai, Yang, Yuxiang, Liu, Ze, Xiong, Yifan, Yang, Ziyue, Ni, Bolin, Hu, Jingcheng, Li, Ruihang, Zhang, Miaosen, Li, Chen, Ning, Jia, Wang, Ruizhe, Zhang, Zheng, Liu, Shuguang, Chau, Joe, Hu, Han, and Cheng, Peng

Subjects
Computer Science - Machine Learning, Computer Science - Computation and Language
Abstract

In this paper, we explore FP8 low-bit data formats for efficient training of large language models (LLMs). Our key insight is that most variables, such as gradients and optimizer states, in LLM training can employ low-precision data formats without compromising model accuracy and requiring no changes to hyper-parameters. Specifically, we propose a new FP8 automatic mixed-precision framework for training LLMs. This framework offers three levels of FP8 utilization to streamline mixed-precision and distributed parallel training for LLMs. It gradually incorporates 8-bit gradients, optimizer states, and distributed learning in an incremental manner. Experiment results show that, during the training of GPT-175B model on H100 GPU platform, our FP8 mixed-precision training framework not only achieved a remarkable 39% reduction in real memory usage but also ran 75% faster than the widely adopted BF16 framework (i.e., Megatron-LM), surpassing the speed of Nvidia Transformer Engine by 37%. This largely reduces the training costs for large foundation models. Furthermore, our FP8 mixed-precision training methodology is generic. It can be seamlessly applied to other tasks such as LLM instruction tuning and reinforcement learning with human feedback, offering savings in fine-tuning expenses. Our FP8 low-precision training framework is open-sourced at {https://github.com/Azure/MS-AMP}{aka.ms/MS.AMP}.

Published
2023

9. NUWA-XL: Diffusion over Diffusion for eXtremely Long Video Generation

Author

Yin, Shengming, Wu, Chenfei, Yang, Huan, Wang, Jianfeng, Wang, Xiaodong, Ni, Minheng, Yang, Zhengyuan, Li, Linjie, Liu, Shuguang, Yang, Fan, Fu, Jianlong, Ming, Gong, Wang, Lijuan, Liu, Zicheng, Li, Houqiang, and Duan, Nan

Subjects
Computer Science - Computer Vision and Pattern Recognition, Computer Science - Artificial Intelligence
Abstract

In this paper, we propose NUWA-XL, a novel Diffusion over Diffusion architecture for eXtremely Long video generation. Most current work generates long videos segment by segment sequentially, which normally leads to the gap between training on short videos and inferring long videos, and the sequential generation is inefficient. Instead, our approach adopts a ``coarse-to-fine'' process, in which the video can be generated in parallel at the same granularity. A global diffusion model is applied to generate the keyframes across the entire time range, and then local diffusion models recursively fill in the content between nearby frames. This simple yet effective strategy allows us to directly train on long videos (3376 frames) to reduce the training-inference gap, and makes it possible to generate all segments in parallel. To evaluate our model, we build FlintstonesHD dataset, a new benchmark for long video generation. Experiments show that our model not only generates high-quality long videos with both global and local coherence, but also decreases the average inference time from 7.55min to 26s (by 94.26\%) at the same hardware setting when generating 1024 frames. The homepage link is \url{https://msra-nuwa.azurewebsites.net/}

Published
2023

10. Effect of Vibrating and Sieving Time on the Reliability of Test Results for Gradation Analysis of Fragile Geomaterials

Author

Yang, Chao, Liu, Shuguang, Bai, Sikai, Zheng, Zheng, Editor-in-Chief, Xi, Zhiyu, Associate Editor, Gong, Siqian, Series Editor, Hong, Wei-Chiang, Series Editor, Mellal, Mohamed Arezki, Series Editor, Narayanan, Ramadas, Series Editor, Nguyen, Quang Ngoc, Series Editor, Ong, Hwai Chyuan, Series Editor, Sun, Zaicheng, Series Editor, Ullah, Sharif, Series Editor, Wu, Junwei, Series Editor, Zhang, Baochang, Series Editor, Zhang, Wei, Series Editor, Zhu, Quanxin, Series Editor, Zheng, Wei, Series Editor, Yuan, Bingxiang, editor, Bilgin, Hüseyin, editor, Luo, Qingzi, editor, and Han, Zejun, editor

Published
2024
Full Text
View/download PDF

11. Information Entropy Theory-Based Optimizing of Gauge Networks for Hydrological Modelling—A Case Study in the Loess Plateau, China

Author

Guo, Yiwei, Han, Haoyu, Nones, Michael, Xu, Wentao, Liu, Shuguang, Rowiński, Paweł M., Editor-in-Chief, Banaszkiewicz, Marek, Series Editor, Węsławski, Jan Marcin, Series Editor, Lewandowski, Marek, Series Editor, Sarna, Marek, Series Editor, Kalinowska, Monika B., editor, and Mrokowska, Magdalena M., editor

Published
2024
Full Text
View/download PDF

12. Overview of Fault Diagnosis Methods for Top Drive System

Author

Liu, Shuguang, Zhang, Guangyong, Wang, Shenghong, Sun, Hao, Angrisani, Leopoldo, Series Editor, Arteaga, Marco, Series Editor, Chakraborty, Samarjit, Series Editor, Chen, Jiming, Series Editor, Chen, Shanben, Series Editor, Chen, Tan Kay, Series Editor, Dillmann, Rüdiger, Series Editor, Duan, Haibin, Series Editor, Ferrari, Gianluigi, Series Editor, Ferre, Manuel, Series Editor, Jabbari, Faryar, Series Editor, Jia, Limin, Series Editor, Kacprzyk, Janusz, Series Editor, Khamis, Alaa, Series Editor, Kroeger, Torsten, Series Editor, Li, Yong, Series Editor, Liang, Qilian, Series Editor, Martín, Ferran, Series Editor, Ming, Tan Cher, Series Editor, Minker, Wolfgang, Series Editor, Misra, Pradeep, Series Editor, Mukhopadhyay, Subhas, Series Editor, Ning, Cun-Zheng, Series Editor, Nishida, Toyoaki, Series Editor, Oneto, Luca, Series Editor, Panigrahi, Bijaya Ketan, Series Editor, Pascucci, Federica, Series Editor, Qin, Yong, Series Editor, Seng, Gan Woon, Series Editor, Speidel, Joachim, Series Editor, Veiga, Germano, Series Editor, Wu, Haitao, Series Editor, Zamboni, Walter, Series Editor, Tan, Kay Chen, Series Editor, Yang, Qingxin, editor, Li, Zewen, editor, and Luo, An, editor

Published
2024
Full Text
View/download PDF

13. Design of Electrical Control System for 4500KN TDS

Author

Liu, Shuguang, Wang, Shenghong, Sun, Hao, Zhang, Guangyong, Angrisani, Leopoldo, Series Editor, Arteaga, Marco, Series Editor, Chakraborty, Samarjit, Series Editor, Chen, Jiming, Series Editor, Chen, Shanben, Series Editor, Chen, Tan Kay, Series Editor, Dillmann, Rüdiger, Series Editor, Duan, Haibin, Series Editor, Ferrari, Gianluigi, Series Editor, Ferre, Manuel, Series Editor, Jabbari, Faryar, Series Editor, Jia, Limin, Series Editor, Kacprzyk, Janusz, Series Editor, Khamis, Alaa, Series Editor, Kroeger, Torsten, Series Editor, Li, Yong, Series Editor, Liang, Qilian, Series Editor, Martín, Ferran, Series Editor, Ming, Tan Cher, Series Editor, Minker, Wolfgang, Series Editor, Misra, Pradeep, Series Editor, Mukhopadhyay, Subhas, Series Editor, Ning, Cun-Zheng, Series Editor, Nishida, Toyoaki, Series Editor, Oneto, Luca, Series Editor, Panigrahi, Bijaya Ketan, Series Editor, Pascucci, Federica, Series Editor, Qin, Yong, Series Editor, Seng, Gan Woon, Series Editor, Speidel, Joachim, Series Editor, Veiga, Germano, Series Editor, Wu, Haitao, Series Editor, Zamboni, Walter, Series Editor, Tan, Kay Chen, Series Editor, Yang, Qingxin, editor, Li, Zewen, editor, and Luo, An, editor

Published
2024
Full Text
View/download PDF

17. Study on the Impact of Marine Science and Technology Talent Agglomeration on Green Total Factor Productivity of Marine Economy

Author

LIU Shuguang and HOU Suqing

Subjects
agglomeration of marine science and technology talents, structural upgrading of the marine industry, green total factor productivity in the marine economy, environmental regulation, mediating effect, Oceanography, GC1-1581
Abstract

Marine science and technology talents are important resources and core elements for promoting the high-quality development of marine economy. This paper constructs a theoretical mechanism of marine science and technology talent agglomeration affecting the green total factor productivity of the marine economy, selects the data of 11 coastal provinces and cities in China from 2006 to 2019 for empirical testing, and further analyzes the mediating role of marine industrial structure upgrading and the moderating role of environmental regulation. The study finds that: there is a significant inverted U-shaped relationship between marine scientific and technological talent agglomeration and the green total factor productivity of the marine economy; there is regional heterogeneity in the impact of marine scientific and technological talent agglomeration on the green total factor productivity of the marine economy, with coastal areas south of the Yangtze River Basin showing a significant inverted U-shaped relationship, while the relationship is insignificant in coastal areas north of the Yangtze River Basin; the upgrading of marine industry structure plays a mediating role in the influence of marine scientific and technological talent agglomeration on the green total factor productivity of the marine economy, and the upgrading of marine industry structure can significantly promote the green total factor productivity of the marine economy; environmental regulation reinforces the inverted U-shaped impact of marine science and technology talent agglomeration on green total factor productivity in the marine economy. Based on this, policy recommendations are put forward to rationally guide the concentration of talents, focus on the matching of talents and industries, and optimize the environment for the development of talents.

Published
2024

42. Assessment of Autonomous Combat Effectiveness of Ground-Attack UAV Based on Optimized Random Forest

Author

Shao Mingjun, Liu Shuguang, Li Shanshan

Subjects
ground-attack uav, combat effectiveness, evaluation index, random forest, weighted mean of vectors algorithm, Motor vehicles. Aeronautics. Astronautics, TL1-4050
Abstract

In order to evaluate the autonomous combat effectiveness of ground-attack UAV efficiently, quickly and objectively, weighted mean of vectors algorithm (INFO) and K-fold cross-validation method are introduced to optimize the random forest algorithm (RF) to find the optimal parameter combination, and an autonomous combat effectiveness evaluation method based on optimized random forest is proposed. Firstly, based on the theory of weighted mean of vectors algorithm, the number and maximum depth of the random forest decision tree model are optimized. Secondly, combined with the ground-attack UAV combat tasks, the main operational factors of autonomous combat effectiveness evalua-tion are analyzed, the evaluation index system of autonomous combat effectiveness of ground-attack UAV is summarized, and the evaluation model of autonomous combat effectiveness of UAV based on INFO-RF is established. Finally, the evaluation model is verified by an example and compared with the traditional RF model, GA-RF model and SVM model. The results show that the output results of INFO-RF model have higher fitting degree and more accurate evaluation value. The results of the example effectively verify the rationality of the proposed method and the reliability of the optimization model.

Published
2023
Full Text
View/download PDF

Catalog

Books, media, physical & digital resources
See catalog results