Search

Your search keyword '"Hu, Longhua"' showing total 762 results
Start Over Author "Hu, Longhua"
762 results on '"Hu, Longhua"'

5. Empower Your Model with Longer and Better Context Comprehension

Author

Gao, Yifei, Wang, Lei, Fang, Jun, Hu, Longhua, and Cheng, Jun

Subjects
Computer Science - Computation and Language, Computer Science - Artificial Intelligence, 68T07, 68T50
Abstract

Recently, with the emergence of numerous Large Language Models (LLMs), the implementation of AI has entered a new era. Irrespective of these models' own capacity and structure, there is a growing demand for LLMs to possess enhanced comprehension of longer and more complex contexts with relatively smaller sizes. Models often encounter an upper limit when processing sequences of sentences that extend beyond their comprehension capacity and result in off-topic or even chaotic responses. While several recent works attempt to address this issue in various ways, they rarely focus on "why models are unable to compensate or strengthen their capabilities on their own". In this paper, we thoroughly investigate the nature of information transfer within LLMs and propose a novel technique called Attention Transition. This technique empowers models to achieve longer and better context comprehension with minimal additional training or impact on generation fluency. Our experiments are conducted on the challenging XSum dataset using LLaMa-7b model with context token length ranging from 800 to 1900. Results demonstrate that we achieve substantial improvements compared with the original generation results evaluated by GPT4., Comment: LLM for long context comprehension

Published
2023

6. Autoencoder and Masked Image Encoding-Based Attentional Pose Network

Author

Hu, Longhua, Ma, Xiaoliang, He, Cheng, Wang, Lei, Cheng, Jun, Goos, Gerhard, Founding Editor, Hartmanis, Juris, Founding Editor, Bertino, Elisa, Editorial Board Member, Gao, Wen, Editorial Board Member, Steffen, Bernhard, Editorial Board Member, Yung, Moti, Editorial Board Member, Liu, Qingshan, editor, Wang, Hanzi, editor, Ma, Zhanyu, editor, Zheng, Weishi, editor, Zha, Hongbin, editor, Chen, Xilin, editor, Wang, Liang, editor, and Ji, Rongrong, editor

Published
2024
Full Text
View/download PDF

Catalog

Books, media, physical & digital resources
See catalog results