LoongTrain: Efficient Training of Long-Sequence LLMs with Head-Context Parallelism

MLA

Gu, Diandian, et al. LoongTrain: Efficient Training of Long-Sequence LLMs with Head-Context Parallelism. 2024. EBSCOhost, widgets.ebscohost.com/prod/customlink/proxify/proxify.php?count=1&encode=0&proxy=&find_1=&replace_1=&target=https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&scope=site&db=edsarx&AN=edsarx.2406.18485&authtype=sso&custid=ns315887.

APA

Gu, D., Sun, P., Hu, Q., Huang, T., Chen, X., Xiong, Y., Wang, G., Chen, Q., Zhao, S., Fang, J., Wen, Y., Zhang, T., Jin, X., & Liu, X. (2024). LoongTrain: Efficient Training of Long-Sequence LLMs with Head-Context Parallelism.

Chicago

Gu, Diandian, Peng Sun, Qinghao Hu, Ting Huang, Xun Chen, Yingtong Xiong, Guoteng Wang, et al. 2024. “LoongTrain: Efficient Training of Long-Sequence LLMs with Head-Context Parallelism.” http://widgets.ebscohost.com/prod/customlink/proxify/proxify.php?count=1&encode=0&proxy=&find_1=&replace_1=&target=https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&scope=site&db=edsarx&AN=edsarx.2406.18485&authtype=sso&custid=ns315887.