Author: "Huang, Yanping" - Searchworks@Jio Institute Digital Library Search Results

Your search keyword '"Huang, Yanping"' showing total 2,739 results

Start Over Author "Huang, Yanping"

2,739 results on '"Huang, Yanping"'

1. Strong Coupling from Inclusive Semi-leptonic Decay of Charmed Mesons

Author: Wu, Jinfei, Lou, Xinchou, Huang, Yanping, Li, Gang, Ye, Jingbo, Che, Yuzhi, and Ruan, Manqi
Subjects: High Energy Physics - Phenomenology, High Energy Physics - Experiment
Abstract: Employing the heavy quark expansion with kinetic scheme, we study the strong interaction coupling $\alpha_S$ at an energy corresponding to the charm quark mass $m_c$ using the data on the semi-leptonic decays of charmed mesons from the BESIII and CLEO-c. The values of the $\alpha_{s}(m_c^{2})$ from the $D^0$, $D^+$, and $D_s^+$ are extracted and found to be consistent. Combining the semileptonic decay widths of the $D^0$, $D^+$, and $D_s^+$, we determine $\alpha_{s}(m_c^{2})$ to be $0.377\pm0.008\pm0.114$, where the first and second uncertainty are the experimental and theoretical uncertainty separately. This result is in good agreement with the world average of $\alpha_S$ running from Z pole to the $m_c$., Comment: 7 pages, 6 figures
Published: 2024

2. DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model

Author: DeepSeek-AI, Liu, Aixin, Feng, Bei, Wang, Bin, Wang, Bingxuan, Liu, Bo, Zhao, Chenggang, Dengr, Chengqi, Ruan, Chong, Dai, Damai, Guo, Daya, Yang, Dejian, Chen, Deli, Ji, Dongjie, Li, Erhang, Lin, Fangyun, Luo, Fuli, Hao, Guangbo, Chen, Guanting, Li, Guowei, Zhang, H., Xu, Hanwei, Yang, Hao, Zhang, Haowei, Ding, Honghui, Xin, Huajian, Gao, Huazuo, Li, Hui, Qu, Hui, Cai, J. L., Liang, Jian, Guo, Jianzhong, Ni, Jiaqi, Li, Jiashi, Chen, Jin, Yuan, Jingyang, Qiu, Junjie, Song, Junxiao, Dong, Kai, Gao, Kaige, Guan, Kang, Wang, Lean, Zhang, Lecong, Xu, Lei, Xia, Leyi, Zhao, Liang, Zhang, Liyue, Li, Meng, Wang, Miaojun, Zhang, Mingchuan, Zhang, Minghua, Tang, Minghui, Li, Mingming, Tian, Ning, Huang, Panpan, Wang, Peiyi, Zhang, Peng, Zhu, Qihao, Chen, Qinyu, Du, Qiushi, Chen, R. J., Jin, R. L., Ge, Ruiqi, Pan, Ruizhe, Xu, Runxin, Chen, Ruyi, Li, S. S., Lu, Shanghao, Zhou, Shangyan, Chen, Shanhuang, Wu, Shaoqing, Ye, Shengfeng, Ma, Shirong, Wang, Shiyu, Zhou, Shuang, Yu, Shuiping, Zhou, Shunfeng, Zheng, Size, Wang, T., Pei, Tian, Yuan, Tian, Sun, Tianyu, Xiao, W. L., Zeng, Wangding, An, Wei, Liu, Wen, Liang, Wenfeng, Gao, Wenjun, Zhang, Wentao, Li, X. Q., Jin, Xiangyue, Wang, Xianzu, Bi, Xiao, Liu, Xiaodong, Wang, Xiaohan, Shen, Xiaojin, Chen, Xiaokang, Chen, Xiaosha, Nie, Xiaotao, Sun, Xiaowen, Wang, Xiaoxiang, Liu, Xin, Xie, Xin, Yu, Xingkai, Song, Xinnan, Zhou, Xinyi, Yang, Xinyu, Lu, Xuan, Su, Xuecheng, Wu, Y., Li, Y. K., Wei, Y. X., Zhu, Y. X., Xu, Yanhong, Huang, Yanping, Li, Yao, Zhao, Yao, Sun, Yaofeng, Li, Yaohui, Wang, Yaohui, Zheng, Yi, Zhang, Yichao, Xiong, Yiliang, Zhao, Yilong, He, Ying, Tang, Ying, Piao, Yishi, Dong, Yixin, Tan, Yixuan, Liu, Yiyuan, Wang, Yongji, Guo, Yongqiang, Zhu, Yuchen, Wang, Yuduan, Zou, Yuheng, Zha, Yukun, Ma, Yunxian, Yan, Yuting, You, Yuxiang, Liu, Yuxuan, Ren, Z. Z., Ren, Zehui, Sha, Zhangli, Fu, Zhe, Huang, Zhen, Zhang, Zhen, Xie, Zhenda, Hao, Zhewen, Shao, Zhihong, Wen, Zhiniu, Xu, Zhipeng, Zhang, Zhongyu, Li, Zhuoshu, Wang, Zihan, Gu, Zihui, Li, Zilin, and Xie, Ziwei
Subjects: Computer Science - Computation and Language, Computer Science - Artificial Intelligence
Abstract: We present DeepSeek-V2, a strong Mixture-of-Experts (MoE) language model characterized by economical training and efficient inference. It comprises 236B total parameters, of which 21B are activated for each token, and supports a context length of 128K tokens. DeepSeek-V2 adopts innovative architectures including Multi-head Latent Attention (MLA) and DeepSeekMoE. MLA guarantees efficient inference through significantly compressing the Key-Value (KV) cache into a latent vector, while DeepSeekMoE enables training strong models at an economical cost through sparse computation. Compared with DeepSeek 67B, DeepSeek-V2 achieves significantly stronger performance, and meanwhile saves 42.5% of training costs, reduces the KV cache by 93.3%, and boosts the maximum generation throughput to 5.76 times. We pretrain DeepSeek-V2 on a high-quality and multi-source corpus consisting of 8.1T tokens, and further perform Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) to fully unlock its potential. Evaluation results show that, even with only 21B activated parameters, DeepSeek-V2 and its chat versions still achieve top-tier performance among open-source models.
Published: 2024

3. Stylus: Automatic Adapter Selection for Diffusion Models

Author: Luo, Michael, Wong, Justin, Trabucco, Brandon, Huang, Yanping, Gonzalez, Joseph E., Chen, Zhifeng, Salakhutdinov, Ruslan, and Stoica, Ion
Subjects: Computer Science - Computer Vision and Pattern Recognition, Computer Science - Artificial Intelligence, Computer Science - Computation and Language, Computer Science - Graphics, Computer Science - Machine Learning
Abstract: Beyond scaling base models with more data or parameters, fine-tuned adapters provide an alternative way to generate high fidelity, custom images at reduced costs. As such, adapters have been widely adopted by open-source communities, accumulating a database of over 100K adapters-most of which are highly customized with insufficient descriptions. This paper explores the problem of matching the prompt to a set of relevant adapters, built on recent work that highlight the performance gains of composing adapters. We introduce Stylus, which efficiently selects and automatically composes task-specific adapters based on a prompt's keywords. Stylus outlines a three-stage approach that first summarizes adapters with improved descriptions and embeddings, retrieves relevant adapters, and then further assembles adapters based on prompts' keywords by checking how well they fit the prompt. To evaluate Stylus, we developed StylusDocs, a curated dataset featuring 75K adapters with pre-computed adapter embeddings. In our evaluation on popular Stable Diffusion checkpoints, Stylus achieves greater CLIP-FID Pareto efficiency and is twice as preferred, with humans and multimodal models as evaluators, over the base model. See stylus-diffusion.github.io for more., Comment: Project Website: https://stylus-diffusion.github.io
Published: 2024

4. Gemini: A Family of Highly Capable Multimodal Models

Author: Gemini Team, Anil, Rohan, Borgeaud, Sebastian, Alayrac, Jean-Baptiste, Yu, Jiahui, Soricut, Radu, Schalkwyk, Johan, Dai, Andrew M., Hauth, Anja, Millican, Katie, Silver, David, Johnson, Melvin, Antonoglou, Ioannis, Schrittwieser, Julian, Glaese, Amelia, Chen, Jilin, Pitler, Emily, Lillicrap, Timothy, Lazaridou, Angeliki, Firat, Orhan, Molloy, James, Isard, Michael, Barham, Paul R., Hennigan, Tom, Lee, Benjamin, Viola, Fabio, Reynolds, Malcolm, Xu, Yuanzhong, Doherty, Ryan, Collins, Eli, Meyer, Clemens, Rutherford, Eliza, Moreira, Erica, Ayoub, Kareem, Goel, Megha, Krawczyk, Jack, Du, Cosmo, Chi, Ed, Cheng, Heng-Tze, Ni, Eric, Shah, Purvi, Kane, Patrick, Chan, Betty, Faruqui, Manaal, Severyn, Aliaksei, Lin, Hanzhao, Li, YaGuang, Cheng, Yong, Ittycheriah, Abe, Mahdieh, Mahdis, Chen, Mia, Sun, Pei, Tran, Dustin, Bagri, Sumit, Lakshminarayanan, Balaji, Liu, Jeremiah, Orban, Andras, Güra, Fabian, Zhou, Hao, Song, Xinying, Boffy, Aurelien, Ganapathy, Harish, Zheng, Steven, Choe, HyunJeong, Weisz, Ágoston, Zhu, Tao, Lu, Yifeng, Gopal, Siddharth, Kahn, Jarrod, Kula, Maciej, Pitman, Jeff, Shah, Rushin, Taropa, Emanuel, Merey, Majd Al, Baeuml, Martin, Chen, Zhifeng, Shafey, Laurent El, Zhang, Yujing, Sercinoglu, Olcan, Tucker, George, Piqueras, Enrique, Krikun, Maxim, Barr, Iain, Savinov, Nikolay, Danihelka, Ivo, Roelofs, Becca, White, Anaïs, Andreassen, Anders, von Glehn, Tamara, Yagati, Lakshman, Kazemi, Mehran, Gonzalez, Lucas, Khalman, Misha, Sygnowski, Jakub, Frechette, Alexandre, Smith, Charlotte, Culp, Laura, Proleev, Lev, Luan, Yi, Chen, Xi, Lottes, James, Schucher, Nathan, Lebron, Federico, Rrustemi, Alban, Clay, Natalie, Crone, Phil, Kocisky, Tomas, Zhao, Jeffrey, Perz, Bartek, Yu, Dian, Howard, Heidi, Bloniarz, Adam, Rae, Jack W., Lu, Han, Sifre, Laurent, Maggioni, Marcello, Alcober, Fred, Garrette, Dan, Barnes, Megan, Thakoor, Shantanu, Austin, Jacob, Barth-Maron, Gabriel, Wong, William, Joshi, Rishabh, Chaabouni, Rahma, Fatiha, Deeni, Ahuja, Arun, Tomar, Gaurav Singh, Senter, Evan, Chadwick, Martin, Kornakov, Ilya, Attaluri, Nithya, Iturrate, Iñaki, Liu, Ruibo, Li, Yunxuan, Cogan, Sarah, Chen, Jeremy, Jia, Chao, Gu, Chenjie, Zhang, Qiao, Grimstad, Jordan, Hartman, Ale Jakse, Garcia, Xavier, Pillai, Thanumalayan Sankaranarayana, Devlin, Jacob, Laskin, Michael, Casas, Diego de Las, Valter, Dasha, Tao, Connie, Blanco, Lorenzo, Badia, Adrià Puigdomènech, Reitter, David, Chen, Mianna, Brennan, Jenny, Rivera, Clara, Brin, Sergey, Iqbal, Shariq, Surita, Gabriela, Labanowski, Jane, Rao, Abhi, Winkler, Stephanie, Parisotto, Emilio, Gu, Yiming, Olszewska, Kate, Addanki, Ravi, Miech, Antoine, Louis, Annie, Teplyashin, Denis, Brown, Geoff, Catt, Elliot, Balaguer, Jan, Xiang, Jackie, Wang, Pidong, Ashwood, Zoe, Briukhov, Anton, Webson, Albert, Ganapathy, Sanjay, Sanghavi, Smit, Kannan, Ajay, Chang, Ming-Wei, Stjerngren, Axel, Djolonga, Josip, Sun, Yuting, Bapna, Ankur, Aitchison, Matthew, Pejman, Pedram, Michalewski, Henryk, Yu, Tianhe, Wang, Cindy, Love, Juliette, Ahn, Junwhan, Bloxwich, Dawn, Han, Kehang, Humphreys, Peter, Sellam, Thibault, Bradbury, James, Godbole, Varun, Samangooei, Sina, Damoc, Bogdan, Kaskasoli, Alex, Arnold, Sébastien M. R., Vasudevan, Vijay, Agrawal, Shubham, Riesa, Jason, Lepikhin, Dmitry, Tanburn, Richard, Srinivasan, Srivatsan, Lim, Hyeontaek, Hodkinson, Sarah, Shyam, Pranav, Ferret, Johan, Hand, Steven, Garg, Ankush, Paine, Tom Le, Li, Jian, Li, Yujia, Giang, Minh, Neitz, Alexander, Abbas, Zaheer, York, Sarah, Reid, Machel, Cole, Elizabeth, Chowdhery, Aakanksha, Das, Dipanjan, Rogozińska, Dominika, Nikolaev, Vitaliy, Sprechmann, Pablo, Nado, Zachary, Zilka, Lukas, Prost, Flavien, He, Luheng, Monteiro, Marianne, Mishra, Gaurav, Welty, Chris, Newlan, Josh, Jia, Dawei, Allamanis, Miltiadis, Hu, Clara Huiyi, de Liedekerke, Raoul, Gilmer, Justin, Saroufim, Carl, Rijhwani, Shruti, Hou, Shaobo, Shrivastava, Disha, Baddepudi, Anirudh, Goldin, Alex, Ozturel, Adnan, Cassirer, Albin, Xu, Yunhan, Sohn, Daniel, Sachan, Devendra, Amplayo, Reinald Kim, Swanson, Craig, Petrova, Dessie, Narayan, Shashi, Guez, Arthur, Brahma, Siddhartha, Landon, Jessica, Patel, Miteyan, Zhao, Ruizhe, Villela, Kevin, Wang, Luyu, Jia, Wenhao, Rahtz, Matthew, Giménez, Mai, Yeung, Legg, Keeling, James, Georgiev, Petko, Mincu, Diana, Wu, Boxi, Haykal, Salem, Saputro, Rachel, Vodrahalli, Kiran, Qin, James, Cankara, Zeynep, Sharma, Abhanshu, Fernando, Nick, Hawkins, Will, Neyshabur, Behnam, Kim, Solomon, Hutter, Adrian, Agrawal, Priyanka, Castro-Ros, Alex, Driessche, George van den, Wang, Tao, Yang, Fan, Chang, Shuo-yiin, Komarek, Paul, McIlroy, Ross, Lučić, Mario, Zhang, Guodong, Farhan, Wael, Sharman, Michael, Natsev, Paul, Michel, Paul, Bansal, Yamini, Qiao, Siyuan, Cao, Kris, Shakeri, Siamak, Butterfield, Christina, Chung, Justin, Rubenstein, Paul Kishan, Agrawal, Shivani, Mensch, Arthur, Soparkar, Kedar, Lenc, Karel, Chung, Timothy, Pope, Aedan, Maggiore, Loren, Kay, Jackie, Jhakra, Priya, Wang, Shibo, Maynez, Joshua, Phuong, Mary, Tobin, Taylor, Tacchetti, Andrea, Trebacz, Maja, Robinson, Kevin, Katariya, Yash, Riedel, Sebastian, Bailey, Paige, Xiao, Kefan, Ghelani, Nimesh, Aroyo, Lora, Slone, Ambrose, Houlsby, Neil, Xiong, Xuehan, Yang, Zhen, Gribovskaya, Elena, Adler, Jonas, Wirth, Mateo, Lee, Lisa, Li, Music, Kagohara, Thais, Pavagadhi, Jay, Bridgers, Sophie, Bortsova, Anna, Ghemawat, Sanjay, Ahmed, Zafarali, Liu, Tianqi, Powell, Richard, Bolina, Vijay, Iinuma, Mariko, Zablotskaia, Polina, Besley, James, Chung, Da-Woon, Dozat, Timothy, Comanescu, Ramona, Si, Xiance, Greer, Jeremy, Su, Guolong, Polacek, Martin, Kaufman, Raphaël Lopez, Tokumine, Simon, Hu, Hexiang, Buchatskaya, Elena, Miao, Yingjie, Elhawaty, Mohamed, Siddhant, Aditya, Tomasev, Nenad, Xing, Jinwei, Greer, Christina, Miller, Helen, Ashraf, Shereen, Roy, Aurko, Zhang, Zizhao, Ma, Ada, Filos, Angelos, Besta, Milos, Blevins, Rory, Klimenko, Ted, Yeh, Chih-Kuan, Changpinyo, Soravit, Mu, Jiaqi, Chang, Oscar, Pajarskas, Mantas, Muir, Carrie, Cohen, Vered, Lan, Charline Le, Haridasan, Krishna, Marathe, Amit, Hansen, Steven, Douglas, Sholto, Samuel, Rajkumar, Wang, Mingqiu, Austin, Sophia, Lan, Chang, Jiang, Jiepu, Chiu, Justin, Lorenzo, Jaime Alonso, Sjösund, Lars Lowe, Cevey, Sébastien, Gleicher, Zach, Avrahami, Thi, Boral, Anudhyan, Srinivasan, Hansa, Selo, Vittorio, May, Rhys, Aisopos, Konstantinos, Hussenot, Léonard, Soares, Livio Baldini, Baumli, Kate, Chang, Michael B., Recasens, Adrià, Caine, Ben, Pritzel, Alexander, Pavetic, Filip, Pardo, Fabio, Gergely, Anita, Frye, Justin, Ramasesh, Vinay, Horgan, Dan, Badola, Kartikeya, Kassner, Nora, Roy, Subhrajit, Dyer, Ethan, Campos, Víctor Campos, Tomala, Alex, Tang, Yunhao, Badawy, Dalia El, White, Elspeth, Mustafa, Basil, Lang, Oran, Jindal, Abhishek, Vikram, Sharad, Gong, Zhitao, Caelles, Sergi, Hemsley, Ross, Thornton, Gregory, Feng, Fangxiaoyu, Stokowiec, Wojciech, Zheng, Ce, Thacker, Phoebe, Ünlü, Çağlar, Zhang, Zhishuai, Saleh, Mohammad, Svensson, James, Bileschi, Max, Patil, Piyush, Anand, Ankesh, Ring, Roman, Tsihlas, Katerina, Vezer, Arpi, Selvi, Marco, Shevlane, Toby, Rodriguez, Mikel, Kwiatkowski, Tom, Daruki, Samira, Rong, Keran, Dafoe, Allan, FitzGerald, Nicholas, Gu-Lemberg, Keren, Khan, Mina, Hendricks, Lisa Anne, Pellat, Marie, Feinberg, Vladimir, Cobon-Kerr, James, Sainath, Tara, Rauh, Maribeth, Hashemi, Sayed Hadi, Ives, Richard, Hasson, Yana, Noland, Eric, Cao, Yuan, Byrd, Nathan, Hou, Le, Wang, Qingze, Sottiaux, Thibault, Paganini, Michela, Lespiau, Jean-Baptiste, Moufarek, Alexandre, Hassan, Samer, Shivakumar, Kaushik, van Amersfoort, Joost, Mandhane, Amol, Joshi, Pratik, Goyal, Anirudh, Tung, Matthew, Brock, Andrew, Sheahan, Hannah, Misra, Vedant, Li, Cheng, Rakićević, Nemanja, Dehghani, Mostafa, Liu, Fangyu, Mittal, Sid, Oh, Junhyuk, Noury, Seb, Sezener, Eren, Huot, Fantine, Lamm, Matthew, De Cao, Nicola, Chen, Charlie, Mudgal, Sidharth, Stella, Romina, Brooks, Kevin, Vasudevan, Gautam, Liu, Chenxi, Chain, Mainak, Melinkeri, Nivedita, Cohen, Aaron, Wang, Venus, Seymore, Kristie, Zubkov, Sergey, Goel, Rahul, Yue, Summer, Krishnakumaran, Sai, Albert, Brian, Hurley, Nate, Sano, Motoki, Mohananey, Anhad, Joughin, Jonah, Filonov, Egor, Kępa, Tomasz, Eldawy, Yomna, Lim, Jiawern, Rishi, Rahul, Badiezadegan, Shirin, Bos, Taylor, Chang, Jerry, Jain, Sanil, Padmanabhan, Sri Gayatri Sundara, Puttagunta, Subha, Krishna, Kalpesh, Baker, Leslie, Kalb, Norbert, Bedapudi, Vamsi, Kurzrok, Adam, Lei, Shuntong, Yu, Anthony, Litvin, Oren, Zhou, Xiang, Wu, Zhichun, Sobell, Sam, Siciliano, Andrea, Papir, Alan, Neale, Robby, Bragagnolo, Jonas, Toor, Tej, Chen, Tina, Anklin, Valentin, Wang, Feiran, Feng, Richie, Gholami, Milad, Ling, Kevin, Liu, Lijuan, Walter, Jules, Moghaddam, Hamid, Kishore, Arun, Adamek, Jakub, Mercado, Tyler, Mallinson, Jonathan, Wandekar, Siddhinita, Cagle, Stephen, Ofek, Eran, Garrido, Guillermo, Lombriser, Clemens, Mukha, Maksim, Sun, Botu, Mohammad, Hafeezul Rahman, Matak, Josip, Qian, Yadi, Peswani, Vikas, Janus, Pawel, Yuan, Quan, Schelin, Leif, David, Oana, Garg, Ankur, He, Yifan, Duzhyi, Oleksii, Älgmyr, Anton, Lottaz, Timothée, Li, Qi, Yadav, Vikas, Xu, Luyao, Chinien, Alex, Shivanna, Rakesh, Chuklin, Aleksandr, Li, Josie, Spadine, Carrie, Wolfe, Travis, Mohamed, Kareem, Das, Subhabrata, Dai, Zihang, He, Kyle, von Dincklage, Daniel, Upadhyay, Shyam, Maurya, Akanksha, Chi, Luyan, Krause, Sebastian, Salama, Khalid, Rabinovitch, Pam G, M, Pavan Kumar Reddy, Selvan, Aarush, Dektiarev, Mikhail, Ghiasi, Golnaz, Guven, Erdem, Gupta, Himanshu, Liu, Boyi, Sharma, Deepak, Shtacher, Idan Heimlich, Paul, Shachi, Akerlund, Oscar, Aubet, François-Xavier, Huang, Terry, Zhu, Chen, Zhu, Eric, Teixeira, Elico, Fritze, Matthew, Bertolini, Francesco, Marinescu, Liana-Eleonora, Bölle, Martin, Paulus, Dominik, Gupta, Khyatti, Latkar, Tejasi, Chang, Max, Sanders, Jason, Wilson, Roopa, Wu, Xuewei, Tan, Yi-Xuan, Thiet, Lam Nguyen, Doshi, Tulsee, Lall, Sid, Mishra, Swaroop, Chen, Wanming, Luong, Thang, Benjamin, Seth, Lee, Jasmine, Andrejczuk, Ewa, Rabiej, Dominik, Ranjan, Vipul, Styrc, Krzysztof, Yin, Pengcheng, Simon, Jon, Harriott, Malcolm Rose, Bansal, Mudit, Robsky, Alexei, Bacon, Geoff, Greene, David, Mirylenka, Daniil, Zhou, Chen, Sarvana, Obaid, Goyal, Abhimanyu, Andermatt, Samuel, Siegler, Patrick, Horn, Ben, Israel, Assaf, Pongetti, Francesco, Chen, Chih-Wei "Louis", Selvatici, Marco, Silva, Pedro, Wang, Kathie, Tolins, Jackson, Guu, Kelvin, Yogev, Roey, Cai, Xiaochen, Agostini, Alessandro, Shah, Maulik, Nguyen, Hung, Donnaile, Noah Ó, Pereira, Sébastien, Friso, Linda, Stambler, Adam, Kuang, Chenkai, Romanikhin, Yan, Geller, Mark, Yan, ZJ, Jang, Kane, Lee, Cheng-Chun, Fica, Wojciech, Malmi, Eric, Tan, Qijun, Banica, Dan, Balle, Daniel, Pham, Ryan, Huang, Yanping, Avram, Diana, Shi, Hongzhi, Singh, Jasjot, Hidey, Chris, Ahuja, Niharika, Saxena, Pranab, Dooley, Dan, Potharaju, Srividya Pranavi, O'Neill, Eileen, Gokulchandran, Anand, Foley, Ryan, Zhao, Kai, Dusenberry, Mike, Liu, Yuan, Mehta, Pulkit, Kotikalapudi, Ragha, Safranek-Shrader, Chalence, Goodman, Andrew, Kessinger, Joshua, Globen, Eran, Kolhar, Prateek, Gorgolewski, Chris, Ibrahim, Ali, Song, Yang, Eichenbaum, Ali, Brovelli, Thomas, Potluri, Sahitya, Lahoti, Preethi, Baetu, Cip, Ghorbani, Ali, Chen, Charles, Crawford, Andy, Pal, Shalini, Sridhar, Mukund, Gurita, Petru, Mujika, Asier, Petrovski, Igor, Cedoz, Pierre-Louis, Li, Chenmei, Chen, Shiyuan, Santo, Niccolò Dal, Goyal, Siddharth, Punjabi, Jitesh, Kappaganthu, Karthik, Kwak, Chester, LV, Pallavi, Velury, Sarmishta, Choudhury, Himadri, Hall, Jamie, Shah, Premal, Figueira, Ricardo, Thomas, Matt, Lu, Minjie, Zhou, Ting, Kumar, Chintu, Jurdi, Thomas, Chikkerur, Sharat, Ma, Yenai, Yu, Adams, Kwak, Soo, Ähdel, Victor, Rajayogam, Sujeevan, Choma, Travis, Liu, Fei, Barua, Aditya, Ji, Colin, Park, Ji Ho, Hellendoorn, Vincent, Bailey, Alex, Bilal, Taylan, Zhou, Huanjie, Khatir, Mehrdad, Sutton, Charles, Rzadkowski, Wojciech, Macintosh, Fiona, Shagin, Konstantin, Medina, Paul, Liang, Chen, Zhou, Jinjing, Shah, Pararth, Bi, Yingying, Dankovics, Attila, Banga, Shipra, Lehmann, Sabine, Bredesen, Marissa, Lin, Zifan, Hoffmann, John Eric, Lai, Jonathan, Chung, Raynald, Yang, Kai, Balani, Nihal, Bražinskas, Arthur, Sozanschi, Andrei, Hayes, Matthew, Alcalde, Héctor Fernández, Makarov, Peter, Chen, Will, Stella, Antonio, Snijders, Liselotte, Mandl, Michael, Kärrman, Ante, Nowak, Paweł, Wu, Xinyi, Dyck, Alex, Vaidyanathan, Krishnan, R, Raghavender, Mallet, Jessica, Rudominer, Mitch, Johnston, Eric, Mittal, Sushil, Udathu, Akhil, Christensen, Janara, Verma, Vishal, Irving, Zach, Santucci, Andreas, Elsayed, Gamaleldin, Davoodi, Elnaz, Georgiev, Marin, Tenney, Ian, Hua, Nan, Cideron, Geoffrey, Leurent, Edouard, Alnahlawi, Mahmoud, Georgescu, Ionut, Wei, Nan, Zheng, Ivy, Scandinaro, Dylan, Jiang, Heinrich, Snoek, Jasper, Sundararajan, Mukund, Wang, Xuezhi, Ontiveros, Zack, Karo, Itay, Cole, Jeremy, Rajashekhar, Vinu, Tumeh, Lara, Ben-David, Eyal, Jain, Rishub, Uesato, Jonathan, Datta, Romina, Bunyan, Oskar, Wu, Shimu, Zhang, John, Stanczyk, Piotr, Zhang, Ye, Steiner, David, Naskar, Subhajit, Azzam, Michael, Johnson, Matthew, Paszke, Adam, Chiu, Chung-Cheng, Elias, Jaume Sanchez, Mohiuddin, Afroz, Muhammad, Faizan, Miao, Jin, Lee, Andrew, Vieillard, Nino, Park, Jane, Zhang, Jiageng, Stanway, Jeff, Garmon, Drew, Karmarkar, Abhijit, Dong, Zhe, Lee, Jong, Kumar, Aviral, Zhou, Luowei, Evens, Jonathan, Isaac, William, Irving, Geoffrey, Loper, Edward, Fink, Michael, Arkatkar, Isha, Chen, Nanxin, Shafran, Izhak, Petrychenko, Ivan, Chen, Zhe, Jia, Johnson, Levskaya, Anselm, Zhu, Zhenkai, Grabowski, Peter, Mao, Yu, Magni, Alberto, Yao, Kaisheng, Snaider, Javier, Casagrande, Norman, Palmer, Evan, Suganthan, Paul, Castaño, Alfonso, Giannoumis, Irene, Kim, Wooyeol, Rybiński, Mikołaj, Sreevatsa, Ashwin, Prendki, Jennifer, Soergel, David, Goedeckemeyer, Adrian, Gierke, Willi, Jafari, Mohsen, Gaba, Meenu, Wiesner, Jeremy, Wright, Diana Gage, Wei, Yawen, Vashisht, Harsha, Kulizhskaya, Yana, Hoover, Jay, Le, Maigo, Li, Lu, Iwuanyanwu, Chimezie, Liu, Lu, Ramirez, Kevin, Khorlin, Andrey, Cui, Albert, LIN, Tian, Wu, Marcus, Aguilar, Ricardo, Pallo, Keith, Chakladar, Abhishek, Perng, Ginger, Abellan, Elena Allica, Zhang, Mingyang, Dasgupta, Ishita, Kushman, Nate, Penchev, Ivo, Repina, Alena, Wu, Xihui, van der Weide, Tom, Ponnapalli, Priya, Kaplan, Caroline, Simsa, Jiri, Li, Shuangfeng, Dousse, Olivier, Piper, Jeff, Ie, Nathan, Pasumarthi, Rama, Lintz, Nathan, Vijayakumar, Anitha, Andor, Daniel, Valenzuela, Pedro, Lui, Minnie, Paduraru, Cosmin, Peng, Daiyi, Lee, Katherine, Zhang, Shuyuan, Greene, Somer, Nguyen, Duc Dung, Kurylowicz, Paula, Hardin, Cassidy, Dixon, Lucas, Janzer, Lili, Choo, Kiam, Feng, Ziqiang, Zhang, Biao, Singhal, Achintya, Du, Dayou, McKinnon, Dan, Antropova, Natasha, Bolukbasi, Tolga, Keller, Orgad, Reid, David, Finchelstein, Daniel, Raad, Maria Abi, Crocker, Remi, Hawkins, Peter, Dadashi, Robert, Gaffney, Colin, Franko, Ken, Bulanova, Anna, Leblond, Rémi, Chung, Shirley, Askham, Harry, Cobo, Luis C., Xu, Kelvin, Fischer, Felix, Xu, Jun, Sorokin, Christina, Alberti, Chris, Lin, Chu-Cheng, Evans, Colin, Dimitriev, Alek, Forbes, Hannah, Banarse, Dylan, Tung, Zora, Omernick, Mark, Bishop, Colton, Sterneck, Rachel, Jain, Rohan, Xia, Jiawei, Amid, Ehsan, Piccinno, Francesco, Wang, Xingyu, Banzal, Praseem, Mankowitz, Daniel J., Polozov, Alex, Krakovna, Victoria, Brown, Sasha, Bateni, MohammadHossein, Duan, Dennis, Firoiu, Vlad, Thotakuri, Meghana, Natan, Tom, Geist, Matthieu, Girgin, Ser tan, Li, Hui, Ye, Jiayu, Roval, Ofir, Tojo, Reiko, Kwong, Michael, Lee-Thorp, James, Yew, Christopher, Sinopalnikov, Danila, Ramos, Sabela, Mellor, John, Sharma, Abhishek, Wu, Kathy, Miller, David, Sonnerat, Nicolas, Vnukov, Denis, Greig, Rory, Beattie, Jennifer, Caveness, Emily, Bai, Libin, Eisenschlos, Julian, Korchemniy, Alex, Tsai, Tomy, Jasarevic, Mimi, Kong, Weize, Dao, Phuong, Zheng, Zeyu, Liu, Frederick, Zhu, Rui, Teh, Tian Huey, Sanmiya, Jason, Gladchenko, Evgeny, Trdin, Nejc, Toyama, Daniel, Rosen, Evan, Tavakkol, Sasan, Xue, Linting, Elkind, Chen, Woodman, Oliver, Carpenter, John, Papamakarios, George, Kemp, Rupert, Kafle, Sushant, Grunina, Tanya, Sinha, Rishika, Talbert, Alice, Wu, Diane, Owusu-Afriyie, Denese, Thornton, Chloe, Pont-Tuset, Jordi, Narayana, Pradyumna, Li, Jing, Fatehi, Saaber, Wieting, John, Ajmeri, Omar, Uria, Benigno, Ko, Yeongil, Knight, Laura, Héliou, Amélie, Niu, Ning, Gu, Shane, Pang, Chenxi, Li, Yeqing, Levine, Nir, Stolovich, Ariel, Santamaria-Fernandez, Rebeca, Goenka, Sonam, Yustalim, Wenny, Strudel, Robin, Elqursh, Ali, Deck, Charlie, Lee, Hyo, Li, Zonglin, Levin, Kyle, Hoffmann, Raphael, Holtmann-Rice, Dan, Bachem, Olivier, Arora, Sho, Koh, Christy, Yeganeh, Soheil Hassas, Põder, Siim, Tariq, Mukarram, Sun, Yanhua, Ionita, Lucian, Seyedhosseini, Mojtaba, Tafti, Pouya, Liu, Zhiyu, Gulati, Anmol, Liu, Jasmine, Ye, Xinyu, Chrzaszcz, Bart, Wang, Lily, Sethi, Nikhil, Li, Tianrun, Brown, Ben, Singh, Shreya, Fan, Wei, Parisi, Aaron, Stanton, Joe, Koverkathu, Vinod, Choquette-Choo, Christopher A., Li, Yunjie, Lu, TJ, Shroff, Prakash, Varadarajan, Mani, Bahargam, Sanaz, Willoughby, Rob, Gaddy, David, Desjardins, Guillaume, Cornero, Marco, Robenek, Brona, Mittal, Bhavishya, Albrecht, Ben, Shenoy, Ashish, Moiseev, Fedor, Jacobsson, Henrik, Ghaffarkhah, Alireza, Rivière, Morgane, Walton, Alanna, Crepy, Clément, Parrish, Alicia, Zhou, Zongwei, Farabet, Clement, Radebaugh, Carey, Srinivasan, Praveen, van der Salm, Claudia, Fidjeland, Andreas, Scellato, Salvatore, Latorre-Chimoto, Eri, Klimczak-Plucińska, Hanna, Bridson, David, de Cesare, Dario, Hudson, Tom, Mendolicchio, Piermaria, Walker, Lexi, Morris, Alex, Mauger, Matthew, Guseynov, Alexey, Reid, Alison, Odoom, Seth, Loher, Lucia, Cotruta, Victor, Yenugula, Madhavi, Grewe, Dominik, Petrushkina, Anastasia, Duerig, Tom, Sanchez, Antonio, Yadlowsky, Steve, Shen, Amy, Globerson, Amir, Webb, Lynette, Dua, Sahil, Li, Dong, Bhupatiraju, Surya, Hurt, Dan, Qureshi, Haroon, Agarwal, Ananth, Shani, Tomer, Eyal, Matan, Khare, Anuj, Belle, Shreyas Rammohan, Wang, Lei, Tekur, Chetan, Kale, Mihir Sanjay, Wei, Jinliang, Sang, Ruoxin, Saeta, Brennan, Liechty, Tyler, Sun, Yi, Zhao, Yao, Lee, Stephan, Nayak, Pandu, Fritz, Doug, Vuyyuru, Manish Reddy, Aslanides, John, Vyas, Nidhi, Wicke, Martin, Ma, Xiao, Eltyshev, Evgenii, Martin, Nina, Cate, Hardie, Manyika, James, Amiri, Keyvan, Kim, Yelin, Xiong, Xi, Kang, Kai, Luisier, Florian, Tripuraneni, Nilesh, Madras, David, Guo, Mandy, Waters, Austin, Wang, Oliver, Ainslie, Joshua, Baldridge, Jason, Zhang, Han, Pruthi, Garima, Bauer, Jakob, Yang, Feng, Mansour, Riham, Gelman, Jason, Xu, Yang, Polovets, George, Liu, Ji, Cai, Honglong, Chen, Warren, Sheng, XiangHai, Xue, Emily, Ozair, Sherjil, Angermueller, Christof, Li, Xiaowei, Sinha, Anoop, Wang, Weiren, Wiesinger, Julia, Koukoumidis, Emmanouil, Tian, Yuan, Iyer, Anand, Gurumurthy, Madhu, Goldenson, Mark, Shah, Parashar, Blake, MK, Yu, Hongkun, Urbanowicz, Anthony, Palomaki, Jennimaria, Fernando, Chrisantha, Durden, Ken, Mehta, Harsh, Momchev, Nikola, Rahimtoroghi, Elahe, Georgaki, Maria, Raul, Amit, Ruder, Sebastian, Redshaw, Morgan, Lee, Jinhyuk, Zhou, Denny, Jalan, Komal, Li, Dinghua, Hechtman, Blake, Schuh, Parker, Nasr, Milad, Milan, Kieran, Mikulik, Vladimir, Franco, Juliana, Green, Tim, Nguyen, Nam, Kelley, Joe, Mahendru, Aroma, Hu, Andrea, Howland, Joshua, Vargas, Ben, Hui, Jeffrey, Bansal, Kshitij, Rao, Vikram, Ghiya, Rakesh, Wang, Emma, Ye, Ke, Sarr, Jean Michel, Preston, Melanie Moranski, Elish, Madeleine, Li, Steve, Kaku, Aakash, Gupta, Jigar, Pasupat, Ice, Juan, Da-Cheng, Someswar, Milan, M., Tejvi, Chen, Xinyun, Amini, Aida, Fabrikant, Alex, Chu, Eric, Dong, Xuanyi, Muthal, Amruta, Buthpitiya, Senaka, Jauhari, Sarthak, Khandelwal, Urvashi, Hitron, Ayal, Ren, Jie, Rinaldi, Larissa, Drath, Shahar, Dabush, Avigail, Jiang, Nan-Jiang, Godhia, Harshal, Sachs, Uli, Chen, Anthony, Fan, Yicheng, Taitelbaum, Hagai, Noga, Hila, Dai, Zhuyun, Wang, James, Hamer, Jenny, Ferng, Chun-Sung, Elkind, Chenel, Atias, Aviel, Lee, Paulina, Listík, Vít, Carlen, Mathias, van de Kerkhof, Jan, Pikus, Marcin, Zaher, Krunoslav, Müller, Paul, Zykova, Sasha, Stefanec, Richard, Gatsko, Vitaly, Hirnschall, Christoph, Sethi, Ashwin, Xu, Xingyu Federico, Ahuja, Chetan, Tsai, Beth, Stefanoiu, Anca, Feng, Bo, Dhandhania, Keshav, Katyal, Manish, Gupta, Akshay, Parulekar, Atharva, Pitta, Divya, Zhao, Jing, Bhatia, Vivaan, Bhavnani, Yashodha, Alhadlaq, Omar, Li, Xiaolin, Danenberg, Peter, Tu, Dennis, Pine, Alex, Filippova, Vera, Ghosh, Abhipso, Limonchik, Ben, Urala, Bhargava, Lanka, Chaitanya Krishna, Clive, Derik, Li, Edward, Wu, Hao, Hongtongsak, Kevin, Li, Ianna, Thakkar, Kalind, Omarov, Kuanysh, Majmundar, Kushal, Alverson, Michael, Kucharski, Michael, Patel, Mohak, Jain, Mudit, Zabelin, Maksim, Pelagatti, Paolo, Kohli, Rohan, Kumar, Saurabh, Kim, Joseph, Sankar, Swetha, Shah, Vineet, Ramachandruni, Lakshmi, Zeng, Xiangkai, Bariach, Ben, Weidinger, Laura, Vu, Tu, Andreev, Alek, He, Antoine, Hui, Kevin, Kashem, Sheleem, Subramanya, Amar, Hsiao, Sissie, Hassabis, Demis, Kavukcuoglu, Koray, Sadovsky, Adam, Le, Quoc, Strohman, Trevor, Wu, Yonghui, Petrov, Slav, Dean, Jeffrey, and Vinyals, Oriol
Subjects: Computer Science - Computation and Language, Computer Science - Artificial Intelligence, Computer Science - Computer Vision and Pattern Recognition
Abstract: This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding. The Gemini family consists of Ultra, Pro, and Nano sizes, suitable for applications ranging from complex reasoning tasks to on-device memory-constrained use-cases. Evaluation on a broad range of benchmarks shows that our most-capable Gemini Ultra model advances the state of the art in 30 of 32 of these benchmarks - notably being the first model to achieve human-expert performance on the well-studied exam benchmark MMLU, and improving the state of the art in every one of the 20 multimodal benchmarks we examined. We believe that the new capabilities of the Gemini family in cross-modal reasoning and language understanding will enable a wide variety of use cases. We discuss our approach toward post-training and deploying Gemini models responsibly to users through services including Gemini, Gemini Advanced, Google AI Studio, and Cloud Vertex AI.
Published: 2023

5. Double Dome and Reemergence of Superconductivity in Pristine 6R-TaS2 under Pressure

Author: Lv, Xindeng, Song, Hao, Chen, Kun, Liu, Sirui, Huang, Yanping, Fang, Yuqiang, and Cui, Tian
Subjects: Condensed Matter - Superconductivity
Abstract: Investigating the implications of interlayer coupling on superconductivity is essential for comprehending the intrinsic mechanisms of high temperature superconductors. Van der Waals heterojunctions have attracted extensive research due to their exotic interlayer coupling. Here, we present a natural heterojunction superconductor of 6R-TaS2 that demonstrates a double-dome of superconductivity, in addition to, the reemergence of superconducting under high pressures. Our first principles calculation shows that the first dome of superconductivity in 6R-TaS2 can be attributed to changes in interlayer coupling and charge transfer. The second superconducting dome and the reemergence of superconductivity can be ascribed to changes in the density of states resulting from Fermi surface reconstruction, in which the DOS of T-layer and S p-orbitals play a crucial role. We have reported the first observation in TMDs that non-metallic atoms playing a dominant role in the reemergence of superconducting and the influence of two Lifshitz transitions on superconducting properties.
Published: 2023

6. Controlled Decoding from Language Models

Author: Mudgal, Sidharth, Lee, Jong, Ganapathy, Harish, Li, YaGuang, Wang, Tao, Huang, Yanping, Chen, Zhifeng, Cheng, Heng-Tze, Collins, Michael, Strohman, Trevor, Chen, Jilin, Beutel, Alex, and Beirami, Ahmad
Subjects: Computer Science - Machine Learning, Computer Science - Artificial Intelligence, Computer Science - Computation and Language
Abstract: KL-regularized reinforcement learning (RL) is a popular alignment framework to control the language model responses towards high reward outcomes. We pose a tokenwise RL objective and propose a modular solver for it, called controlled decoding (CD). CD exerts control through a separate prefix scorer module, which is trained to learn a value function for the reward. The prefix scorer is used at inference time to control the generation from a frozen base model, provably sampling from a solution to the RL objective. We empirically demonstrate that CD is effective as a control mechanism on popular benchmarks. We also show that prefix scorers for multiple rewards may be combined at inference time, effectively solving a multi-objective RL problem with no additional training. We show that the benefits of applying CD transfer to an unseen base model with no further tuning as well. Finally, we show that CD can be applied in a blockwise decoding fashion at inference-time, essentially bridging the gap between the popular best-of-K strategy and tokenwise control through reinforcement learning. This makes CD a promising approach for alignment of language models., Comment: ICML 2024
Published: 2023

7. SPAE: Semantic Pyramid AutoEncoder for Multimodal Generation with Frozen LLMs

Author: Yu, Lijun, Cheng, Yong, Wang, Zhiruo, Kumar, Vivek, Macherey, Wolfgang, Huang, Yanping, Ross, David A., Essa, Irfan, Bisk, Yonatan, Yang, Ming-Hsuan, Murphy, Kevin, Hauptmann, Alexander G., and Jiang, Lu
Subjects: Computer Science - Computer Vision and Pattern Recognition, Computer Science - Computation and Language, Computer Science - Multimedia
Abstract: In this work, we introduce Semantic Pyramid AutoEncoder (SPAE) for enabling frozen LLMs to perform both understanding and generation tasks involving non-linguistic modalities such as images or videos. SPAE converts between raw pixels and interpretable lexical tokens (or words) extracted from the LLM's vocabulary. The resulting tokens capture both the semantic meaning and the fine-grained details needed for visual reconstruction, effectively translating the visual content into a language comprehensible to the LLM, and empowering it to perform a wide array of multimodal tasks. Our approach is validated through in-context learning experiments with frozen PaLM 2 and GPT 3.5 on a diverse set of image understanding and generation tasks. Our method marks the first successful attempt to enable a frozen LLM to generate image content while surpassing state-of-the-art performance in image understanding tasks, under the same setting, by over 25%., Comment: NeurIPS 2023 spotlight
Published: 2023

8. Self-Assembled Homopolymeric Spherulites from Small Molecules in Solution

Author: Song, Qiantao, Li, Yi, Jin, Zhicheng, Liu, Hai, Creyer, Matthew N, Yim, Wonjun, Huang, Yanping, Hu, Xiaobing, He, Tengyu, Li, Yajuan, Kelley, Shana O, Shi, Lingyan, Zhou, Jiajing, and Jokerst, Jesse V
Subjects: Macromolecular and Materials Chemistry, Chemical Sciences, Bioengineering, Nanotechnology, Crystallization, Polymers, Freezing, Nanoparticles, General Chemistry, Chemical sciences, Engineering
Abstract: Polymeric spherulites are typically formed by melt crystallization: spherulitic growth in solution is rare and requires complex polymers and dilute solutions. Here, we report the mild and unique formation of luminescent spherulites at room temperature via the simple molecule benzene-1,4-dithiol (BDT). Specifically, BDT polymerized into oligomers (PBDT) via disulfide bonds and assembled into uniform supramolecular nanoparticles in aqueous buffer; these nanoparticles were then dissolved back into PBDT in a good solvent (i.e., dimethylformamide) and underwent chain elongation to form spherulites (rPBDT) in 10 min. The spherulite geometry was modulated by changing the PBDT concentration and reaction time. Due to the step-growth polymerization and reorganization of PBDT, these spherulites not only exhibited robust structure but also showed broad clusterization-triggered emission. The biocompatibility and efficient cellular uptake of the spherulites further underscore their value as traceable drug carriers. This system provides a new pathway for designing versatile superstructures with value for hierarchical assembly of small molecules into a complicated biological system.
Published: 2023

9. Brainformers: Trading Simplicity for Efficiency

Author: Zhou, Yanqi, Du, Nan, Huang, Yanping, Peng, Daiyi, Lan, Chang, Huang, Da, Shakeri, Siamak, So, David, Dai, Andrew, Lu, Yifeng, Chen, Zhifeng, Le, Quoc, Cui, Claire, Laudon, James, and Dean, Jeff
Subjects: Computer Science - Machine Learning, Computer Science - Computation and Language
Abstract: Transformers are central to recent successes in natural language processing and computer vision. Transformers have a mostly uniform backbone where layers alternate between feed-forward and self-attention in order to build a deep network. Here we investigate this design choice and find that more complex blocks that have different permutations of layer primitives can be more efficient. Using this insight, we develop a complex block, named Brainformer, that consists of a diverse sets of layers such as sparsely gated feed-forward layers, dense feed-forward layers, attention layers, and various forms of layer normalization and activation functions. Brainformer consistently outperforms the state-of-the-art dense and sparse Transformers, in terms of both quality and efficiency. A Brainformer model with 8 billion activated parameters per token demonstrates 2x faster training convergence and 5x faster step time compared to its GLaM counterpart. In downstream task evaluation, Brainformer also demonstrates a 3% higher SuperGLUE score with fine-tuning compared to GLaM with a similar number of activated parameters. Finally, Brainformer largely outperforms a Primer dense model derived with NAS with similar computation per token on fewshot evaluations.
Published: 2023

10. Lifelong Language Pretraining with Distribution-Specialized Experts

Author: Chen, Wuyang, Zhou, Yanqi, Du, Nan, Huang, Yanping, Laudon, James, Chen, Zhifeng, and Cu, Claire
Subjects: Computer Science - Computation and Language, Computer Science - Machine Learning
Abstract: Pretraining on a large-scale corpus has become a standard method to build general language models (LMs). Adapting a model to new data distributions targeting different downstream tasks poses significant challenges. Naive fine-tuning may incur catastrophic forgetting when the over-parameterized LMs overfit the new data but fail to preserve the pretrained features. Lifelong learning (LLL) aims to enable information systems to learn from a continuous data stream across time. However, most prior work modifies the training recipe assuming a static fixed network architecture. We find that additional model capacity and proper regularization are key elements to achieving strong LLL performance. Thus, we propose Lifelong-MoE, an extensible MoE (Mixture-of-Experts) architecture that dynamically adds model capacity via adding experts with regularized pretraining. Our results show that by only introducing a limited number of extra experts while keeping the computation cost constant, our model can steadily adapt to data distribution shifts while preserving the previous knowledge. Compared to existing lifelong learning approaches, Lifelong-MoE achieves better few-shot performance on 19 downstream NLP tasks., Comment: ICML 2023
Published: 2023

11. PaLM 2 Technical Report

Author: Anil, Rohan, Dai, Andrew M., Firat, Orhan, Johnson, Melvin, Lepikhin, Dmitry, Passos, Alexandre, Shakeri, Siamak, Taropa, Emanuel, Bailey, Paige, Chen, Zhifeng, Chu, Eric, Clark, Jonathan H., Shafey, Laurent El, Huang, Yanping, Meier-Hellstern, Kathy, Mishra, Gaurav, Moreira, Erica, Omernick, Mark, Robinson, Kevin, Ruder, Sebastian, Tay, Yi, Xiao, Kefan, Xu, Yuanzhong, Zhang, Yujing, Abrego, Gustavo Hernandez, Ahn, Junwhan, Austin, Jacob, Barham, Paul, Botha, Jan, Bradbury, James, Brahma, Siddhartha, Brooks, Kevin, Catasta, Michele, Cheng, Yong, Cherry, Colin, Choquette-Choo, Christopher A., Chowdhery, Aakanksha, Crepy, Clément, Dave, Shachi, Dehghani, Mostafa, Dev, Sunipa, Devlin, Jacob, Díaz, Mark, Du, Nan, Dyer, Ethan, Feinberg, Vlad, Feng, Fangxiaoyu, Fienber, Vlad, Freitag, Markus, Garcia, Xavier, Gehrmann, Sebastian, Gonzalez, Lucas, Gur-Ari, Guy, Hand, Steven, Hashemi, Hadi, Hou, Le, Howland, Joshua, Hu, Andrea, Hui, Jeffrey, Hurwitz, Jeremy, Isard, Michael, Ittycheriah, Abe, Jagielski, Matthew, Jia, Wenhao, Kenealy, Kathleen, Krikun, Maxim, Kudugunta, Sneha, Lan, Chang, Lee, Katherine, Lee, Benjamin, Li, Eric, Li, Music, Li, Wei, Li, YaGuang, Li, Jian, Lim, Hyeontaek, Lin, Hanzhao, Liu, Zhongtao, Liu, Frederick, Maggioni, Marcello, Mahendru, Aroma, Maynez, Joshua, Misra, Vedant, Moussalem, Maysam, Nado, Zachary, Nham, John, Ni, Eric, Nystrom, Andrew, Parrish, Alicia, Pellat, Marie, Polacek, Martin, Polozov, Alex, Pope, Reiner, Qiao, Siyuan, Reif, Emily, Richter, Bryan, Riley, Parker, Ros, Alex Castro, Roy, Aurko, Saeta, Brennan, Samuel, Rajkumar, Shelby, Renee, Slone, Ambrose, Smilkov, Daniel, So, David R., Sohn, Daniel, Tokumine, Simon, Valter, Dasha, Vasudevan, Vijay, Vodrahalli, Kiran, Wang, Xuezhi, Wang, Pidong, Wang, Zirui, Wang, Tao, Wieting, John, Wu, Yuhuai, Xu, Kelvin, Xu, Yunhan, Xue, Linting, Yin, Pengcheng, Yu, Jiahui, Zhang, Qiao, Zheng, Steven, Zheng, Ce, Zhou, Weikang, Zhou, Denny, Petrov, Slav, and Wu, Yonghui
Subjects: Computer Science - Computation and Language, Computer Science - Artificial Intelligence
Abstract: We introduce PaLM 2, a new state-of-the-art language model that has better multilingual and reasoning capabilities and is more compute-efficient than its predecessor PaLM. PaLM 2 is a Transformer-based model trained using a mixture of objectives. Through extensive evaluations on English and multilingual language, and reasoning tasks, we demonstrate that PaLM 2 has significantly improved quality on downstream tasks across different model sizes, while simultaneously exhibiting faster and more efficient inference compared to PaLM. This improved efficiency enables broader deployment while also allowing the model to respond faster, for a more natural pace of interaction. PaLM 2 demonstrates robust reasoning capabilities exemplified by large improvements over PaLM on BIG-Bench and other reasoning tasks. PaLM 2 exhibits stable performance on a suite of responsible AI evaluations, and enables inference-time control over toxicity without additional overhead or impact on other capabilities. Overall, PaLM 2 achieves state-of-the-art performance across a diverse set of tasks and capabilities. When discussing the PaLM 2 family, it is important to distinguish between pre-trained models (of various sizes), fine-tuned variants of these models, and the user-facing products that use these models. In particular, user-facing products typically include additional pre- and post-processing steps. Additionally, the underlying models may evolve over time. Therefore, one should not expect the performance of user-facing products to exactly match the results reported in this report.
Published: 2023

12. Design Comparison for the Supercritical CO2 Brayton Cycle with Recompression and Thermal Regeneration: Numerical Results

Author: Chen, Jiaxiang, Chen, Lin, Zang, Jinguang, Huang, Yanping, and Chen, Lin, editor
Published: 2024
Full Text: View/download PDF

13. A zincophilic molecular brush for a dendrite-free, corrosion-resistant, zinc metal anode with a long life cycle

Author: Liu, Penggao, Guo, Jia, Chen, Xinyue, Wang, Ting, Huang, Yanping, Gao, Shasha, Wang, Tao, Wu, Dongling, and Liu, Kaiyu
Published: 2024
Full Text: View/download PDF

14. AlpaServe: Statistical Multiplexing with Model Parallelism for Deep Learning Serving

Author: Li, Zhuohan, Zheng, Lianmin, Zhong, Yinmin, Liu, Vincent, Sheng, Ying, Jin, Xin, Huang, Yanping, Chen, Zhifeng, Zhang, Hao, Gonzalez, Joseph E., and Stoica, Ion
Subjects: Computer Science - Machine Learning, Computer Science - Distributed, Parallel, and Cluster Computing, Computer Science - Networking and Internet Architecture
Abstract: Model parallelism is conventionally viewed as a method to scale a single large deep learning model beyond the memory limits of a single device. In this paper, we demonstrate that model parallelism can be additionally used for the statistical multiplexing of multiple devices when serving multiple models, even when a single model can fit into a single device. Our work reveals a fundamental trade-off between the overhead introduced by model parallelism and the opportunity to exploit statistical multiplexing to reduce serving latency in the presence of bursty workloads. We explore the new trade-off space and present a novel serving system, AlpaServe, that determines an efficient strategy for placing and parallelizing collections of large deep learning models across a distributed cluster. Evaluation results on production workloads show that AlpaServe can process requests at up to 10x higher rates or 6x more burstiness while staying within latency constraints for more than 99% of requests., Comment: OSDI 2023
Published: 2023

15. Massively Multilingual Shallow Fusion with Large Language Models

Author: Hu, Ke, Sainath, Tara N., Li, Bo, Du, Nan, Huang, Yanping, Dai, Andrew M., Zhang, Yu, Cabrera, Rodrigo, Chen, Zhifeng, and Strohman, Trevor
Subjects: Computer Science - Computation and Language, Computer Science - Machine Learning
Abstract: While large language models (LLM) have made impressive progress in natural language processing, it remains unclear how to utilize them in improving automatic speech recognition (ASR). In this work, we propose to train a single multilingual language model (LM) for shallow fusion in multiple languages. We push the limits of the multilingual LM to cover up to 84 languages by scaling up using a mixture-of-experts LLM, i.e., generalist language model (GLaM). When the number of experts increases, GLaM dynamically selects only two at each decoding step to keep the inference computation roughly constant. We then apply GLaM to a multilingual shallow fusion task based on a state-of-the-art end-to-end model. Compared to a dense LM of similar computation during inference, GLaM reduces the WER of an English long-tail test set by 4.4% relative. In a multilingual shallow fusion task, GLaM improves 41 out of 50 languages with an average relative WER reduction of 3.85%, and a maximum reduction of 10%. Compared to the baseline model, GLaM achieves an average WER reduction of 5.53% over 43 languages., Comment: Accepted to IEEE ICASSP 2023
Published: 2023

16. Effect of Composting Treatment of Grape Branches on Wine Quality

Author: TU Qian, DING Weikang, HUANG Yanping, LI Yuanyuan, LIU Shiqiu, MIN Xuwu, ZHENG Changhua, YUAN Chunlong
Subjects: grape branches, compost, wine, aroma, color, Food processing and manufacture, TP368-456
Abstract: In order to study the effect of applying different proportions of compost made from grape branches in the vineyard on the quality of wine, 8-year-old ‘Pinot Noir’ grapevine was selected for this study, and compost made from 100% crushed grape branches (GB), compost made from 70% crushed grape branches + 30% sheep manure (GM1), compost made from 50% crushed grape branches + 50% sheep manure (GM2), compost made from 30% crushed grape branches + 70% sheep manure (GM3), and 100% sheep manure (SM) were applied to the vineyard , and the no compost addition treatment (CK) was used as a control. One year after fertilization, grape berries were taken to brew wine, and the basic physicochemical indexes of grapes and wine, the phenolic substances, color, and aroma substances and sensory evaluation of wine were determined. The results showed that the composting treatment significantly increased the contents of alcohol, total phenols, total anthocyanins, total flavan-3-ols, and total flavonoids in Pinot Noir wine, with the GM3 treatment showing the greatest increase in the levels of alcohol and titratable acids, by 16.5% and 24.03%, respectively. In terms of color, the composting treatments increased the color saturation of ‘Pinot Noir’ wine. Orthogonal partial least squares-discriminant analysis (OPLS-DA) of aroma substances showed that the major differential aroma substances in ‘Pinot Noir’ wine among different composting treatments were isoamyl alcohol, followed by ethyl acetate, 1-propanol, and phenylethanol, and that composting treatments with the addition of grape branches increased the levels of ethyl acetate, isoamyl acetate, ethyl caproate, and ethyl caproate in ‘Pinot Noir’ wine. In terms of sensory quality, the intensity of astringency, bitterness, and sweetness of ‘Pinot Noir’ wine was enhanced by the composting treatments, and the wine was improved in terms of taste and balance. This study provides a theoretical basis for the comprehensive utilization of grape branch waste and the improvement of wine quality.
Published: 2024
Full Text: View/download PDF

17. Expected $H\to\mu^{+}\mu^{-}$ measurement precision with $e^{+}e^{-}\to Z(q\bar{q})H$ production at the CEPC

Author: Liu, Qi, Ran, Kunlin, Huang, Yanping, Li, Gang, Ruan, Manqi, Jin, Shan, and Sun, Liang
Subjects: High Energy Physics - Experiment
Abstract: A search for the dimuon decay of the Standard Model Higgs boson is performed using the Monte Carlo simulated events to mimic data corresponding to an integrated luminosity of 5.6 ab$^{-1}$ collected with the Circular Electron-Positron Collider detector in $e^{+}e^{-}$ collisions at $\sqrt{s}=240$ GeV. The paper studies $e^{+}e^{-}\to ZH,\,Z\to q\bar{q},\,H\to\mu^{+}\mu^{-}$ process, and the expected significance considering only the data statistical uncertainty over the background-only hypothesis for a Higgs boson with a mass of 125 GeV is found to be 6.1$\sigma$, corresponding to the precision of 19%. The systematic impacts from the background Monte Carlo statistical fluctuations are estimated to be negligible. The dependence of the measurement accuracy on the muon momentum resolution of the CEPC detector has been investigated. It is found that the muon momentum resolution has to be better than 204 MeV to discover the $H\to\mu\mu$ process at the nominal integrated luminosity. And if the resolution is 100% worse than the designed parameter, the integrated luminosity is needed to be greater than 7.2 ab$^{-1}$ to reach 5$\sigma$ significance., Comment: 10 pages, 8 figures, submitted to Chinese Physics C
Published: 2022
Full Text: View/download PDF

18. Scaling Instruction-Finetuned Language Models

Author: Chung, Hyung Won, Hou, Le, Longpre, Shayne, Zoph, Barret, Tay, Yi, Fedus, William, Li, Yunxuan, Wang, Xuezhi, Dehghani, Mostafa, Brahma, Siddhartha, Webson, Albert, Gu, Shixiang Shane, Dai, Zhuyun, Suzgun, Mirac, Chen, Xinyun, Chowdhery, Aakanksha, Castro-Ros, Alex, Pellat, Marie, Robinson, Kevin, Valter, Dasha, Narang, Sharan, Mishra, Gaurav, Yu, Adams, Zhao, Vincent, Huang, Yanping, Dai, Andrew, Yu, Hongkun, Petrov, Slav, Chi, Ed H., Dean, Jeff, Devlin, Jacob, Roberts, Adam, Zhou, Denny, Le, Quoc V., and Wei, Jason
Subjects: Computer Science - Machine Learning, Computer Science - Computation and Language
Abstract: Finetuning language models on a collection of datasets phrased as instructions has been shown to improve model performance and generalization to unseen tasks. In this paper we explore instruction finetuning with a particular focus on (1) scaling the number of tasks, (2) scaling the model size, and (3) finetuning on chain-of-thought data. We find that instruction finetuning with the above aspects dramatically improves performance on a variety of model classes (PaLM, T5, U-PaLM), prompting setups (zero-shot, few-shot, CoT), and evaluation benchmarks (MMLU, BBH, TyDiQA, MGSM, open-ended generation). For instance, Flan-PaLM 540B instruction-finetuned on 1.8K tasks outperforms PALM 540B by a large margin (+9.4% on average). Flan-PaLM 540B achieves state-of-the-art performance on several benchmarks, such as 75.2% on five-shot MMLU. We also publicly release Flan-T5 checkpoints, which achieve strong few-shot performance even compared to much larger models, such as PaLM 62B. Overall, instruction finetuning is a general method for improving the performance and usability of pretrained language models., Comment: Public checkpoints: https://huggingface.co/docs/transformers/model_doc/flan-t5
Published: 2022

19. Building Machine Translation Systems for the Next Thousand Languages

Author: Bapna, Ankur, Caswell, Isaac, Kreutzer, Julia, Firat, Orhan, van Esch, Daan, Siddhant, Aditya, Niu, Mengmeng, Baljekar, Pallavi, Garcia, Xavier, Macherey, Wolfgang, Breiner, Theresa, Axelrod, Vera, Riesa, Jason, Cao, Yuan, Chen, Mia Xu, Macherey, Klaus, Krikun, Maxim, Wang, Pidong, Gutkin, Alexander, Shah, Apurva, Huang, Yanping, Chen, Zhifeng, Wu, Yonghui, and Hughes, Macduff
Subjects: Computer Science - Computation and Language, Computer Science - Artificial Intelligence, Computer Science - Machine Learning
Abstract: In this paper we share findings from our effort to build practical machine translation (MT) systems capable of translating across over one thousand languages. We describe results in three research domains: (i) Building clean, web-mined datasets for 1500+ languages by leveraging semi-supervised pre-training for language identification and developing data-driven filtering techniques; (ii) Developing practical MT models for under-served languages by leveraging massively multilingual models trained with supervised parallel data for over 100 high-resource languages and monolingual datasets for an additional 1000+ languages; and (iii) Studying the limitations of evaluation metrics for these languages and conducting qualitative analysis of the outputs from our MT models, highlighting several frequent error modes of these types of models. We hope that our work provides useful insights to practitioners working towards building MT systems for currently understudied languages, and highlights research directions that can complement the weaknesses of massively multilingual models in data-sparse settings., Comment: V2: updated with some details from 24-language Google Translate launch in May 2022 V3: spelling corrections, additional acknowledgements
Published: 2022

20. Antitumor effects of erlotinib in combination with berberine in A431 cells

Author: Cuan, Xiangdan, Yang, Xingying, Zhu, Weiwei, Zhao, Yue, Luo, Rui, Huang, Yanping, Wang, Xuanjun, and Sheng, Jun
Published: 2023
Full Text: View/download PDF

21. Mixture-of-Experts with Expert Choice Routing

Author: Zhou, Yanqi, Lei, Tao, Liu, Hanxiao, Du, Nan, Huang, Yanping, Zhao, Vincent, Dai, Andrew, Chen, Zhifeng, Le, Quoc, and Laudon, James
Subjects: Computer Science - Machine Learning, Computer Science - Artificial Intelligence
Abstract: Sparsely-activated Mixture-of-experts (MoE) models allow the number of parameters to greatly increase while keeping the amount of computation for a given token or a given sample unchanged. However, a poor expert routing strategy (e.g. one resulting in load imbalance) can cause certain experts to be under-trained, leading to an expert being under or over-specialized. Prior work allocates a fixed number of experts to each token using a top-k function regardless of the relative importance of different tokens. To address this, we propose a heterogeneous mixture-of-experts employing an expert choice method. Instead of letting tokens select the top-k experts, we have experts selecting the top-k tokens. As a result, each token can be routed to a variable number of experts and each expert can have a fixed bucket size. We systematically study pre-training speedups using the same computational resources of the Switch Transformer top-1 and GShard top-2 gating of prior work and find that our method improves training convergence time by more than 2x. For the same computational cost, our method demonstrates higher performance in fine-tuning 11 selected tasks in the GLUE and SuperGLUE benchmarks. For a smaller activation cost, our method outperforms the T5 dense model in 7 out of the 11 tasks.
Published: 2022

22. ST-MoE: Designing Stable and Transferable Sparse Expert Models

Author: Zoph, Barret, Bello, Irwan, Kumar, Sameer, Du, Nan, Huang, Yanping, Dean, Jeff, Shazeer, Noam, and Fedus, William
Subjects: Computer Science - Computation and Language, Computer Science - Machine Learning
Abstract: Scale has opened new frontiers in natural language processing -- but at a high cost. In response, Mixture-of-Experts (MoE) and Switch Transformers have been proposed as an energy efficient path to even larger and more capable language models. But advancing the state-of-the-art across a broad set of natural language tasks has been hindered by training instabilities and uncertain quality during fine-tuning. Our work focuses on these issues and acts as a design guide. We conclude by scaling a sparse model to 269B parameters, with a computational cost comparable to a 32B dense encoder-decoder Transformer (Stable and Transferable Mixture-of-Experts or ST-MoE-32B). For the first time, a sparse model achieves state-of-the-art performance in transfer learning, across a diverse set of tasks including reasoning (SuperGLUE, ARC Easy, ARC Challenge), summarization (XSum, CNN-DM), closed book question answering (WebQA, Natural Questions), and adversarially constructed tasks (Winogrande, ANLI R3)., Comment: 25 pages main text, 39 pages overall
Published: 2022

23. Alpa: Automating Inter- and Intra-Operator Parallelism for Distributed Deep Learning

Author: Zheng, Lianmin, Li, Zhuohan, Zhang, Hao, Zhuang, Yonghao, Chen, Zhifeng, Huang, Yanping, Wang, Yida, Xu, Yuanzhong, Zhuo, Danyang, Xing, Eric P., Gonzalez, Joseph E., and Stoica, Ion
Subjects: Computer Science - Machine Learning, Computer Science - Distributed, Parallel, and Cluster Computing, Computer Science - Programming Languages
Abstract: Alpa automates model-parallel training of large deep learning (DL) models by generating execution plans that unify data, operator, and pipeline parallelism. Existing model-parallel training systems either require users to manually create a parallelization plan or automatically generate one from a limited space of model parallelism configurations. They do not suffice to scale out complex DL models on distributed compute devices. Alpa distributes the training of large DL models by viewing parallelisms as two hierarchical levels: inter-operator and intra-operator parallelisms. Based on it, Alpa constructs a new hierarchical space for massive model-parallel execution plans. Alpa designs a number of compilation passes to automatically derive efficient parallel execution plans at each parallelism level. Alpa implements an efficient runtime to orchestrate the two-level parallel execution on distributed compute devices. Our evaluation shows Alpa generates parallelization plans that match or outperform hand-tuned model-parallel training systems even on models they are designed for. Unlike specialized systems, Alpa also generalizes to models with heterogeneous architectures and models without manually-designed plans. Alpa's source code is publicly available at https://github.com/alpa-projects/alpa, Comment: OSDI 2022
Published: 2022

24. LaMDA: Language Models for Dialog Applications

Author: Thoppilan, Romal, De Freitas, Daniel, Hall, Jamie, Shazeer, Noam, Kulshreshtha, Apoorv, Cheng, Heng-Tze, Jin, Alicia, Bos, Taylor, Baker, Leslie, Du, Yu, Li, YaGuang, Lee, Hongrae, Zheng, Huaixiu Steven, Ghafouri, Amin, Menegali, Marcelo, Huang, Yanping, Krikun, Maxim, Lepikhin, Dmitry, Qin, James, Chen, Dehao, Xu, Yuanzhong, Chen, Zhifeng, Roberts, Adam, Bosma, Maarten, Zhao, Vincent, Zhou, Yanqi, Chang, Chung-Ching, Krivokon, Igor, Rusch, Will, Pickett, Marc, Srinivasan, Pranesh, Man, Laichee, Meier-Hellstern, Kathleen, Morris, Meredith Ringel, Doshi, Tulsee, Santos, Renelito Delos, Duke, Toju, Soraker, Johnny, Zevenbergen, Ben, Prabhakaran, Vinodkumar, Diaz, Mark, Hutchinson, Ben, Olson, Kristen, Molina, Alejandra, Hoffman-John, Erin, Lee, Josh, Aroyo, Lora, Rajakumar, Ravi, Butryna, Alena, Lamm, Matthew, Kuzmina, Viktoriya, Fenton, Joe, Cohen, Aaron, Bernstein, Rachel, Kurzweil, Ray, Aguera-Arcas, Blaise, Cui, Claire, Croak, Marian, Chi, Ed, and Le, Quoc
Subjects: Computer Science - Computation and Language, Computer Science - Artificial Intelligence
Abstract: We present LaMDA: Language Models for Dialog Applications. LaMDA is a family of Transformer-based neural language models specialized for dialog, which have up to 137B parameters and are pre-trained on 1.56T words of public dialog data and web text. While model scaling alone can improve quality, it shows less improvements on safety and factual grounding. We demonstrate that fine-tuning with annotated data and enabling the model to consult external knowledge sources can lead to significant improvements towards the two key challenges of safety and factual grounding. The first challenge, safety, involves ensuring that the model's responses are consistent with a set of human values, such as preventing harmful suggestions and unfair bias. We quantify safety using a metric based on an illustrative set of human values, and we find that filtering candidate responses using a LaMDA classifier fine-tuned with a small amount of crowdworker-annotated data offers a promising approach to improving model safety. The second challenge, factual grounding, involves enabling the model to consult external knowledge sources, such as an information retrieval system, a language translator, and a calculator. We quantify factuality using a groundedness metric, and we find that our approach enables the model to generate responses grounded in known sources, rather than responses that merely sound plausible. Finally, we explore the use of LaMDA in the domains of education and content recommendations, and analyze their helpfulness and role consistency.
Published: 2022

25. GLaM: Efficient Scaling of Language Models with Mixture-of-Experts

Author: Du, Nan, Huang, Yanping, Dai, Andrew M., Tong, Simon, Lepikhin, Dmitry, Xu, Yuanzhong, Krikun, Maxim, Zhou, Yanqi, Yu, Adams Wei, Firat, Orhan, Zoph, Barret, Fedus, Liam, Bosma, Maarten, Zhou, Zongwei, Wang, Tao, Wang, Yu Emma, Webster, Kellie, Pellat, Marie, Robinson, Kevin, Meier-Hellstern, Kathleen, Duke, Toju, Dixon, Lucas, Zhang, Kun, Le, Quoc V, Wu, Yonghui, Chen, Zhifeng, and Cui, Claire
Subjects: Computer Science - Computation and Language
Abstract: Scaling language models with more data, compute and parameters has driven significant progress in natural language processing. For example, thanks to scaling, GPT-3 was able to achieve strong results on in-context learning tasks. However, training these large dense models requires significant amounts of computing resources. In this paper, we propose and develop a family of language models named GLaM (Generalist Language Model), which uses a sparsely activated mixture-of-experts architecture to scale the model capacity while also incurring substantially less training cost compared to dense variants. The largest GLaM has 1.2 trillion parameters, which is approximately 7x larger than GPT-3. It consumes only 1/3 of the energy used to train GPT-3 and requires half of the computation flops for inference, while still achieving better overall zero-shot and one-shot performance across 29 NLP tasks., Comment: Accepted to ICML 2022
Published: 2021

26. Safety and feasibility of third-party cytotoxic T lymphocytes for high-risk patients with COVID-19

Author: Grosso, Dolores, Wagner, John L., O’Connor, Allyson, Keck, Kaitlyn, Huang, Yanping, Wang, Zi-Xuan, Mehler, Hilary, Leiby, Benjamin, Flomenberg, Phyllis, Gergis, Usama, Nikbakht, Neda, Morris, Michael, Karp, Julie, Peedin, Alexis, and Flomenberg, Neal
Published: 2024
Full Text: View/download PDF

27. Experimental study on single- to two-phase flow and heat transfer of water in a small tube

Author: Hong, Guangwei, Qu, Fanyu, Liu, Shenghui, Gao, Jiaying, Wu, Yihang, Liu, Hanxing, Yuan, Feixiang, Gao, Ruiying, Chen, Liangyong, Zhu, Xiaoliang, and Huang, Yanping
Published: 2024
Full Text: View/download PDF

28. Measurement and comparison of transient thermal boundary layer of a Local-Heated Mini-Channel under sub- and super-critical conditions

Author: Zeng, Gang, Chen, Lin, Yang, Dong, Yuan, Haizhuan, Zang, Jinguang, and Huang, Yanping
Published: 2024
Full Text: View/download PDF

29. Research on prediction and analysis of supercritical water heat transfer coefficient based on support vector machine

Author: Ma Dongliang, Li Yi, Zhou Tao, and Huang Yanping
Subjects: Supercritical water, Support vector machine, Radial basis kernel function, Data normalization, Heat transfer coefficient, Machine learning algorithm, Nuclear engineering. Atomic power, TK9001-9401
Abstract: In order to better perform thermal hydraulic calculation and analysis of supercritical water reactor, based on the experimental data of supercritical water, the model training and predictive analysis of the heat transfer coefficient of supercritical water were carried out by using the support vector machine (SVM) algorithm. The changes in the prediction accuracy of the supercritical water heat transfer coefficient are analyzed by the changes of the regularization penalty parameter C, the slack variable epsilon and the Gaussian kernel function parameter gamma. The predicted value of the SVM model obtained after parameter optimization and the actual experimental test data are analyzed for data verification. The research results show that: the normalization of the data has a great influence on the prediction results. The slack variable has a relatively small influence on the accuracy change range of the predicted heat transfer coefficient. The change of gamma has the greatest impact on the accuracy of the heat transfer coefficient. Compared with the calculation results of traditional empirical formula methods, the trained algorithm model using SVM has smaller average error and standard deviations. Using the SVM trained algorithm model, the heat transfer coefficient of supercritical water can be effectively predicted and analyzed.
Published: 2023
Full Text: View/download PDF

30. Initiation and Development of Supercritical Carbon Dioxide Nuclear Energy and Power System

Author: HUANG Yanping;LIU Minyun;ZHUO Wenbin;YE Lü;TANG Jia;CHEN Yaoxing;LIU Ruilong;LIU Xiuting;TANG Yu;ZHAO Xuebin;GONG Houjun;ZAN Yuanfeng
Subjects: supercritical carbon dioxide, brayton cycle, nuclear reactor, nuclear energy and power system, Nuclear engineering. Atomic power, TK9001-9401, Nuclear and particle physics. Atomic energy. Radioactivity, QC770-798
Abstract: The supercritical carbon dioxide (S-CO2) nuclear energy and power system uses S-CO2 as the working fluid to convert nuclear heat directly or indirectly into electricity or mechanical energy. S-CO2 is considered one of the most promising working fluids for applications in conventional and high-temperature scenarios. The unique properties of S-CO2, such as its non-phase-change behavior, high density, compressibility near the pseudo-critical region, low thermal cycle compression ratio, minimal thermal inertia equipment requirements, and suitability for direct cycle and power generation propulsion technologies, simplify the system, reduce volume, lower weight, improve maneuverability, reduce noise, and enhance thermal efficiency. The advantages of S-CO2 power conversion technology not only meet the development needs of the next generation of advanced energy conversion technology but also align with the future trends of nuclear power systems. Leading energy technology countries, including the United States, conducted extensive research on S-CO2 power conversion technology in areas such as fossil fuel power generation, nuclear energy, waste heat recovery, and solar thermal power. Some of these projects have already entered the engineering demonstration phase. In China, S-CO2 power conversion technology was included in the “14th Five-Year Plan” for scientific and technological innovation in the energy sector. Numerous universities and industrial sectors in China are actively conducting research and development in this field. The development of S-CO2 nuclear power system in three historical stages for nearly 60 years, including the concept creation, research restart, and collaborative innovation, was summarized in this paper. The research status of S-CO2 nuclear energy and power system in reactor design, material technology, thermohydraulics, heat exchangers, turbomachines, system operation, control, and security policies was also introduced. The technical challenges and key research directions faced by S-CO2 nuclear energy and power system in basic research and engineering were pointed out. Currently, energy superpowers represented by China and the United States have completed laboratory tests of S-CO2 power conversion systems, and are expected to move towards medium-scale engineering demonstration or even large-scale application in the next 5 to 10 years. S-CO2 nuclear energy and power system has the prerequisites for engineering development, and is expected to lead the transformation of advanced nuclear energy technologies. Nevertheless, numerous technical challenges are still existing in core design, materials, thermal-fluid mechanics, and safety analysis. Additionally, engineering efforts are required to tackle challenges related to high-efficiency heat exchangers, highpower high-speed electric motors, heavy-duty electromagnetic bearings, and turbine power generation systems. Simultaneously, coordinated efforts in both fundamental and applied research are essential to enhance the advanced nature and engineering feasibility of S-CO2 nuclear energy and power systems.
Published: 2023
Full Text: View/download PDF

31. Optimization Design Research of Supercritical Carbon Dioxide Power Cycle under Low-temperature Heat Sink Environment

Author: WANG Dianle;HUANG Yanping;YIN Kaikai;LIU Minyun;ZHOU Yuan;LAI Xiangpeng;LI Yong
Subjects: supercritical carbon dioxide power cycle, low-temperature heat sink, cold-end optimization, Nuclear engineering. Atomic power, TK9001-9401, Nuclear and particle physics. Atomic energy. Radioactivity, QC770-798
Abstract: This paper presents a cold-end optimization design study of a nuclear energy power conversion system in the context of low-temperature heat sink applications. Based on the principles of simple recuperation cycles and recompression cycles, the study focuses on the supercritical carbon dioxide power cycle and its cold-end optimization for application in low-temperature heat sink environments. These strategies are designed to enhance the efficiency and reduce the operating pressure of the supercritical carbon dioxide power cycle under low-temperature heat sink conditions. Three different cold-end optimization approaches were proposed, which include: a non-condensing liquid-phase compressed supercritical carbon dioxide power cycle, a condensing liquid-phase compressed supercritical carbon dioxide power cycle, and a gas-phase compressed supercritical carbon dioxide power cycle. This research established a thermodynamic analysis program, which modeled turbine machinery such as compressors and expanders using isentropic models. Heat exchangers such as recuperations and coolers were simplified and modeled, with pressure drops and end temperature differences assumed. The study’s thermodynamic analysis program was compared with computational results from a Sandia National Laboratories research report, confirming the accuracy of the thermodynamic analysis methodology presented in this paper. For a 100 MW scale power generation unit, the thermodynamic analysis program was used to conduct computational analyses on both simple recuperation cycles and recompression cycles. These cycles were driven by a medium-to-high-temperature reactor heat source with a turbine inlet temperature of 550 ℃, and a medium-to-low-temperature reactor heat source with a turbine inlet temperature of 315 ℃. The study reveals that the optimal cold-end optimization approach varies based on the heat source temperature and cycle configuration. For the medium-to-high-temperature reactor heat source, the gasphase compressed supercritical carbon dioxide power cycle in the simple recuperation cycle configuration achieves the highest efficiency at 39%. In the recompression cycle configuration, the condensing supercritical carbon dioxide power cycle achieves the highest efficiency at 48%. For the medium-to-low-temperature reactor heat source, regardless of the cycle type, the condensing supercritical carbon dioxide power cycle yields the highest efficiency, the highest efficiency is 28% for the simple recuperation cycle and 33% for the recompression cycle. When comparing the cold-end optimized supercritical carbon dioxide power cycle to conventional supercritical carbon dioxide power cycles with the same maximum temperature and pressure, the optimized system exhibits a reduction in operating pressure drop and an efficiency improvement of 2% to 4%. The cold-end optimized configurations of the supercritical carbon dioxide power cycle hold promising applications for nuclear energy power conversion systems with low-temperature heat sink environments.
Published: 2023
Full Text: View/download PDF

32. Analysis of Brayton Cycle Characteristics of CO2-SF6 Mixed Working Fluid

Author: XI Dapeng;LIU Minyun;LIU Xiuting;FEI Junjie;ZANG Jinguang;HUANG Yanping
Subjects: supercritical carbon dioxide, mixed working fluid, thermophysical property calculation, cycle efficiency, Nuclear engineering. Atomic power, TK9001-9401, Nuclear and particle physics. Atomic energy. Radioactivity, QC770-798
Abstract: The supercritical carbon dioxide (S-CO2) Brayton cycle is an efficient energy conversion system. It has the advantages of simple and compact, low noise and high efficiency. Therefore, it has attracted much attention from academia and industry in recent years. However, the current S-CO2 Brayton cycle study usually considers the application scenarios with ambient temperature 20 ℃, and the minimum cyclic temperature is 32 ℃. The adaptive analysis and optimization of low temperature cold source scenarios such as deep sea and polar regions are not optimized. CO2-SF6 mixture has lower critical parameters than pure CO2, so that it is expected to further reduce the compression power, improve the turbine pressure ratio, and thus improve the cycle efficiency with low ambient temperature. In this paper, the PR equation and van der Waals mixing rules were used to calculate the CO2-SF6 mixture thermal properties with the Nishiumi interaction coefficient model. The thermal properties calculated are in good agreement with the experiment data in literature. Based on the calculated thermal properties, a simple recuperation cycle was analyzed with a maximum pressure of 12 MPa, a minimum cyclic temperature of 15-55 ℃, and a minimum pressure slightly higher than the critical pressure. The results show that replacing the pure CO2 with a CO2-SF6 mixture can effectively improve the turbine pressure ratio, thereby improving the cycle efficiency, and reducing the critical temperature and the optimal compressor inlet temperature. And the use of CO2-SF6 mixture can effectively decrease cycle efficiency degradation when the compressor inlet temperature is lower than the quasicritical temperature, thereby improving the stability of the cycle under the low temperature cold source. The preliminary analysis results show that the use of CO2-SF6 mixed working fluid instead of pure CO2 working fluid can effectively improve the performance of the cycle in the low temperature environment, which also indicates that the mixed working fluid scheme is one of the effective solutions to expand the applicable temperature range of the S-CO2 Brayton cycle.
Published: 2023
Full Text: View/download PDF

33. Study on Void Reactivity of Supercritical Carbon Dioxide Cooled Reactor

Author: LIU Minyun;CUI Rongyi;ZHAO Xingyu;HAN Wenbin;HUANG Shanfang;HUANG Yanping
Subjects: supercritical carbon dioxide reactor, loss of coolant accident, void reactivity, gas-cooled fast reactor, Nuclear engineering. Atomic power, TK9001-9401, Nuclear and particle physics. Atomic energy. Radioactivity, QC770-798
Abstract: Due to the weak moderating capability of carbon dioxide itself, supercritical carbon dioxide cooled reactor typically has a hard energy spectrum and is susceptible to safety issues introduced by positive reactivity feedback in the event of LOCA (loss of coolant accident). In this paper, based on the energy spectrum characteristics of supercritical carbon dioxide reactor, a three-factor formula was proposed to describe the neutron circulation process. Based on the analysis of the threefactor formula, the effective fission neutron number and neutron utilization factor were refered as spectral terms in this study, while the probability of no leakage was referred as the leakage term. When studying the effect of moderator, fuel burnup, etc. on the void reactivity, the energy spectrum factor was mainly considered. When studying the effect of radial reflector material, high radial ratio, etc. on the void reactivity, the leakage factor was mainly considered, providing guidance for the core design of supercritical carbon dioxide reactors based on this theoretical basis. Using Monte Carlo simulation method, modeling, verification, and analysis were carried out for the supercritical carbon dioxide cooled reactor design proposed by MIT. The fuel assemblies of this reactor adopt a hexagonal annular fuel design, with the center of the assembly being the SCO2 coolant channel, which is separated from the MOX fuel by a cladding. A total of 265 fuel rod assemblies arranged in a hexagonal pattern make up a fuel component, and 562 fuel components (divided into three zones) and 21 control rod components arranged in a hexagonal pattern make up the reactor core active zone. Axial and radial reflector layers are set outside the active zone, and the outermost layer is a B4C absorber. The height of the active zone is 1.54 m, and the effective diameter is 4.81 m, with a height-to-diameter ratio of H/D=0.32 and a thermal power output of 2 400 MW. The results show that the design of supercritical carbon dioxide reactor needs to focus on the softening of energy spectrum and reasonable core geometry design. Using supercritical carbon dioxide as the radial reflector layer, in the event of LOCA, the carbon dioxide as the reflector and the carbon dioxide inside the core will leak simultaneously, and the increase in core leakage rate will lead to a decrease in void reactivity. Compared with other materials such as BeO, PbO, and SiO2, using a carbon dioxide reflector layer can better ensure the criticality of the core and improve neutron economy. However, due to the generally low neutron reflection ability of supercritical carbon dioxide itself, if the core has a large radial leakage due to a high core-to-diameter ratio, a thicker radial reflector layer is required, which may bring additional engineering difficulties and costs, requiring balance in the design. Setting moderators by partition can flatten the neutron flux, soften the energy spectrum, and reduce the reactivity introduced by LOCA.
Published: 2023
Full Text: View/download PDF

34. Experimental Study on Heat Transfer Performance of Supercritical CO2-water in Complex Mini-channels

Author: JIN Feng;CHEN Deqi;HU Lian;HUANG Yanping
Subjects: supercritical co2, brayton cycle, printed circuit heat exchanger, zigzag channel, Nuclear engineering. Atomic power, TK9001-9401, Nuclear and particle physics. Atomic energy. Radioactivity, QC770-798
Abstract: The printed circuit heat exchanger (PCHE) is one of the promising candidates for supercritical CO2 Brayton cycle, its performance is important to the safety and economy of the next generation nuclear technology. In this paper, an experimental study on heat transfer performance of supercritical CO2-water in zigzag mini-channels was conducted to explore the heat transfer characteristics in the pre-cooler channels of a supercritical CO2 Brayton cycle. The experimental pressure of the supercritical CO2 is 7.5-9.0 MPa, the temperature is 50-90 ℃, and the flow rate is 300-600 kg/(m2·s). The test section has a diameter of 2.0 mm, a pitch of 7.24 mm, and a bend angle of 40°. By arranging multiple temperature measurement points, the temperature distribution data along the complex mini channels were obtained. The influence of different operating conditions on the logarithmic mean temperature difference and overall heat transfer coefficient along the mini channels was studied, and the effects of overall method and segmented method on the calculation results were further evaluated. The results show that the overall heat transfer coefficient exhibits a relatively high level near the pseudo-critical region. Meanwhile, due to the pronounced enhancement effect of the buffer layer on the near-critical temperature, the peak value of the overall heat transfer coefficient corresponds to a slightly higher CO2 bulk temperature than the near-critical temperature. Under the given operating conditions, increasing the CO2 inlet temperature has a small effect on the overall heat transfer coefficient of the heat exchanger. With the increase of inlet pressure, the peak value of overall heat transfer coefficient decreases, the peak values of the overall heat transfer coefficient corresponding to 7.5, 8.0, and 9.0 MPa are 3 828.3, 3 466.0, and 3 281.8 W/(m2·℃) respectively. However, the overall trend gradually becomes less sensitive to the changes in bulk temperature and exhibits higher heat transfer capability in the gas-like region. Increasing the inlet mass flow rate can significantly improve the overall heat transfer capacity of the heat exchanger. When the inlet mass flux increases from 311.72 kg/(m2·s) to 495.37 kg/(m2·s), the maximum value of the overall heat transfer coefficient increases from 3 305 W/(m2·℃) to 4 193 W/(m2·℃), which is a 26.87% improvement. Besides, in the heat transfer conditions near the pseudocritical region, the deviation between the overall heat transfer coefficients calculated by the overall method and the segmented method is significant, while in the heat transfer conditions across the pseudocritical region, due to the compensating effect of the s-shaped logarithmic mean temperature difference, the deviation in the calculation results is significantly reduced.
Published: 2023
Full Text: View/download PDF

35. BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognition

Author: Zhang, Yu, Park, Daniel S., Han, Wei, Qin, James, Gulati, Anmol, Shor, Joel, Jansen, Aren, Xu, Yuanzhong, Huang, Yanping, Wang, Shibo, Zhou, Zongwei, Li, Bo, Ma, Min, Chan, William, Yu, Jiahui, Wang, Yongqiang, Cao, Liangliang, Sim, Khe Chai, Ramabhadran, Bhuvana, Sainath, Tara N., Beaufays, Françoise, Chen, Zhifeng, Le, Quoc V., Chiu, Chung-Cheng, Pang, Ruoming, and Wu, Yonghui
Subjects: Electrical Engineering and Systems Science - Audio and Speech Processing, Computer Science - Computation and Language, Computer Science - Machine Learning, Computer Science - Sound
Abstract: We summarize the results of a host of efforts using giant automatic speech recognition (ASR) models pre-trained using large, diverse unlabeled datasets containing approximately a million hours of audio. We find that the combination of pre-training, self-training and scaling up model size greatly increases data efficiency, even for extremely large tasks with tens of thousands of hours of labeled data. In particular, on an ASR task with 34k hours of labeled data, by fine-tuning an 8 billion parameter pre-trained Conformer model we can match state-of-the-art (SoTA) performance with only 3% of the training data and significantly improve SoTA with the full training set. We also report on the universal benefits gained from using big pre-trained and self-trained models for a large set of downstream tasks that cover a wide range of speech domains and span multiple orders of magnitudes of dataset sizes, including obtaining SoTA performance on many public benchmarks. In addition, we utilize the learned representation of pre-trained networks to achieve SoTA results on non-ASR tasks., Comment: 14 pages, 7 figures, 13 tables; v2: minor corrections, reference baselines and bibliography updated; v3: corrections based on reviewer feedback, bibliography updated
Published: 2021
Full Text: View/download PDF

36. Beyond Distillation: Task-level Mixture-of-Experts for Efficient Inference

Author: Kudugunta, Sneha, Huang, Yanping, Bapna, Ankur, Krikun, Maxim, Lepikhin, Dmitry, Luong, Minh-Thang, and Firat, Orhan
Subjects: Computer Science - Computation and Language, Computer Science - Machine Learning
Abstract: Sparse Mixture-of-Experts (MoE) has been a successful approach for scaling multilingual translation models to billions of parameters without a proportional increase in training computation. However, MoE models are prohibitively large and practitioners often resort to methods such as distillation for serving. In this work, we investigate routing strategies at different granularity (token, sentence, task) in MoE models to bypass distillation. Experiments on WMT and a web-scale dataset suggest that task-level routing (task-MoE) enables us to extract smaller, ready-to-deploy sub-networks from large sparse models. On WMT, our task-MoE with 32 experts (533M parameters) outperforms the best performing token-level MoE model (token-MoE) by +1.0 BLEU on average across 30 language pairs. The peak inference throughput is also improved by a factor of 1.9x when we route by tasks instead of tokens. While distilling a token-MoE to a smaller dense model preserves only 32% of the BLEU gains, our sub-network task-MoE, by design, preserves all the gains with the same inference cost as the distilled student model. Finally, when scaling up to 200 language pairs, our 128-expert task-MoE (13B parameters) performs competitively with a token-level counterpart, while improving the peak inference throughput by a factor of 2.6x., Comment: EMNLP Findings 2021
Published: 2021

37. A review of supercritical fluid flow instability

Author: Luo, Qiao, Zhou, Yuan, Huang, Jiajian, Hu, Wei, Yuan, Yuan, and Huang, Yanping
Published: 2024
Full Text: View/download PDF

38. Conceptual design and optimization of heat rejection system for nuclear reactor coupled with supercritical carbon dioxide Brayton cycle used on Mars

Author: Qiao, Jianxin, Liu, Shenghui, Liu, Minyun, Huang, Yanping, Yuan, Feixiang, Zhu, Xiaoliang, and Cui, Yinghuan
Published: 2024
Full Text: View/download PDF

39. Optimize the design analysis of hybrid fin structure microchannel heat exchanger

Author: Yanying, Wu, Guo, Zhangpeng, Huang, Yanping, Lu, Yiming, Gao, Chong, Guo, Wentao, Lv, Haicai, Liu, Yang, and Niu, Fenglei
Published: 2024
Full Text: View/download PDF

40. A numerical method for two-dimensional transient nonlinear convection-diffusion equations

Author: Meng, Xiangyuan, Huang, Mei, Wang, Boxue, Ouyang, Xiaoping, Huang, Yanping, Chen, Denggao, Cheng, Yanting, and Li, Yaodi
Published: 2024
Full Text: View/download PDF

41. Pool heat transfer studies of near-critical and supercritical carbon dioxide

Author: Liu, Minyun, Liu, Shenghui, Huang, Yanping, Zheng, Ruohan, Yuan, Haohan, Wang, Jie, Gong, Houjun, Wang, Yanlin, and Huang, Shanfang
Published: 2024
Full Text: View/download PDF

42. Biomass-derived polymer as a flexible “zincophilic–hydrophobic” solid electrolyte interphase layer to enable practical Zn metal anodes

Author: Chen, Wenjian, Tan, Yi, Guo, Chengyue, Zhang, Xiaoyan, He, Xin, Kuang, Wei, Weng, Haofan, Du, He, Huang, Dan, Huang, Yanping, Xu, Jing, and He, Huibing
Published: 2024
Full Text: View/download PDF

43. Quantitative visualization on boundary heat transfer of supercritical CO2 mini-channel through-flow (Re=104) via pixelated phase-shifting interferometry

Author: Chen, Lin, Zeng, Gang, Yang, Dong, Yuan, Haizhuan, Zang, Jinguang, and Huang, Yanping
Published: 2024
Full Text: View/download PDF

44. CCL20 and CD8A as potential diagnostic biomarkers for HBV-induced liver fibrosis in chronic hepatitis B

Author: Song, Jingru, Liu, Lu, Wang, Zheng, Xie, Dong, Azami, Nisma Lena Bahaji, Lu, Lu, Huang, Yanping, Ye, Wei, Zhang, Qin, and Sun, Mingyu
Published: 2024
Full Text: View/download PDF

45. A new procedure for solving multigroup neutron diffusion eigenvalue problems

Author: Cheng, YanTing, Huang, Mei, Ouyang, XiaoPing, Huang, YanPing, Chen, DengGao, and Matsuda, Hiroshi
Published: 2024
Full Text: View/download PDF

46. Syntheses and coordination of linear and macrocyclic pentapyrroles possessing a fused moiety: Confusion modulated structural diversity

Author: Huang, Yanping, Zhu, Bin, Li, Qizhao, Baryshnikov, Glib, Li, Chengjie, Sha, Feng, Wu, Xin-Yan, Ågren, Hans, and Xie, Yongshu
Published: 2024
Full Text: View/download PDF

47. Direct numerical simulation of flow and heat transfer of supercritical water with different heat fluxes

Author: Bai, Yifan, Wang, Han, Liu, Minyun, Wu, Jinghui, Lyu, Haicai, and Huang, Yanping
Published: 2024
Full Text: View/download PDF

48. Effects of wastewater treatment plant effluent on microbial risks of pathogens and their antibiotic resistance in the receiving river

Author: Xiao, Zijian, Qin, Yuanyuan, Han, Li, Liu, Yifan, Wang, Ziyi, Huang, Yanping, Ma, Yujing, and Zou, Yina
Published: 2024
Full Text: View/download PDF

49. Numerical study on a 7-pin bundle fuel assembly with different wire arrangements

Author: Li, Yaodi, Huang, Mei, Ouyang, Xiaoping, and Huang, Yanping
Published: 2024
Full Text: View/download PDF

50. Experimental study of the depressurization phenomena of supercritical carbon dioxide system

Author: Liu, Minyun, Huang, Yanping, Wang, Yangle, Zheng, Ruohan, Fei, Junjie, Tian, Fengyuan, Gong, Houjun, and Zhuo, Wenbin
Published: 2024
Full Text: View/download PDF

Catalog

Books, media, physical & digital resources

See catalog results

Searchworks

Select search scope, currently: Articles Catalog books, media & more in Jio Institute collections Articles journal articles & other e-resources

Search

Search Constraints

Refine your results

Search Limiters

Topic

Publication Year Range

Language

Publication Type

Journal

Region

Database

Publisher

2,739 results on '"Huang, Yanping"'

Search Results

Catalog

Select search scope, currently: Articles

Catalog

books, media & more in Jio Institute collections

Articles

journal articles & other e-resources