Search

Your search keyword '"Sébastien M"' showing total 1,180 results
Start Over Author "Sébastien M"
1,180 results on '"Sébastien M"'

1. Reaching the precision limit with tensor-based wavefront shaping

Author

Rodrigo Gutiérrez-Cuevas, Dorian Bouchet, Julien de Rosny, and Sébastien M. Popoff

Subjects
Science
Abstract

Abstract Perturbations in complex media, due to their own dynamical evolution or to external effects, are often seen as detrimental. Therefore, a common strategy, especially for telecommunication and imaging applications, is to limit the sensitivity to those perturbations in order to avoid them. Here, instead, we consider enhancing the interaction between light and perturbations to produce the largest change in the output intensity distribution. Our work hinges on the use of tensor-based techniques, presently at the forefront of machine learning explorations, to study intensity-based measurements where its quadratic relationship to the field prevents the use of standard matrix methods. With this tensor-based framework, we can identify the maximum-information intensity channel which maximizes the change in its output intensity distribution and the Fisher information encoded in it about a given perturbation. We further demonstrate experimentally its superiority for robust and precise sensing applications. Additionally, we derive the appropriate strategy to reach the precision limit for intensity-based measurements, leading to an increase in Fisher information by more than four orders of magnitude compared to the mean for random wavefronts when measured with the pixels of a camera.

Published
2024
Full Text
View/download PDF

2. Predictors of blood ionized calcium concentration in sick adult cattle

Author

Tolga Karapinar, Kenan Cagri Tumer, Peter D. Constable, and Sébastien M. C. Buczinski

Subjects
blood pH, hypochloremia, ionized calcium, Veterinary medicine, SF600-1100
Abstract

Abstract Background Data on the factors affecting blood ionized calcium concentration (ciCa2+) and diagnostic performance of serum total calcium concentration (ctCa) measurements to detect abnormal blood iCa2+ status are lacking in sick adult cattle. Objective Assess the association of ciCa2+ with venous blood pH, plasma concentrations of chloride (cCl), sodium (cNa), and potassium (cK), and ctCa, and total protein, albumin, and globulin concentrations in sick adult cattle. Animals Two‐hundred and sixty‐five adult cattle (≥1‐year‐old) with different diseases. Methods Prospective study. Whole blood pH, ciCa2+, cNa, cK, and cCl were measured using a blood gas and electrolyte analyzer, whereas ctCa, and total protein, and albumin concentrations were determined using an autoanalyzer. The relationship between ciCa2+ and venous blood pH, plasma cCl, cNa, cK, and ctCa, and total protein, albumin, and globulin concentrations was investigated. Sensitivity and specificity were calculated for ctCa for diagnosis of abnormal ciCa2+. Results Sensitivity of ctCa measurements to detect abnormal ciCa2+ was 66.0% whereas specificity of ctCa measurements was 72.3%. Serum total calcium concentration measurements accounted for 42% of adjusted blood ionized calcium (iCa2+7.40) concentration variance. Plasma cCl, and cK had explanatory power of ciCa2+7.40, accounting for an additional 21% and 9% of the variance, respectively. Conclusions and Clinical Importance Serum tCa measurements failed to accurately predict blood iCa2+ status in ill adult cattle. Serum tCa concentrations and plasma cCl were the strongest predictors of ciCa2+ in sick adult cattle.

Published
2024
Full Text
View/download PDF

3. Characterization and Exploitation of the Rotational Memory Effect in Multimode Fibers

Author

Rodrigo Gutiérrez-Cuevas, Arthur Goetschy, Yaron Bromberg, Guy Pelc, Esben Ravn Andresen, Laurent Bigot, Yves Quiquempois, Maroun Bsaibes, Pierre Sillard, Marianne Bigot, Ori Katz, Julien de Rosny, and Sébastien M. Popoff

Subjects
Physics, QC1-999
Abstract

In an ideal perfectly straight multimode fiber with a circular core, the symmetry ensures that rotating the input wave front leads to a corresponding rotation of the output wave front. This invariant property, known as the rotational memory effect (RME), remains independent of the typically unknown output profile. The RME thus offers significant potential for imaging and telecommunication applications. However, in real-life fibers, this effect is degraded by intrinsic imperfections and external perturbations, and is challenging to observe because of its acute sensitivity to misalignments and aberrations in the optical setup. Building on a previously established method for precisely estimating fiber transmission properties, we demonstrate an accurate extraction of RME properties. Additionally, we introduce a comprehensive theoretical framework for both qualitative and quantitative analysis, which specifically links the angular-dependent correlation of the RME to the core deformation’s geometrical properties and the fiber’s mode characteristics. Our theoretical predictions align well with experimental data and simulations for various amounts of fiber distorsion. Finally, we demonstrate the ability to engineer wave fronts with significantly enhanced correlation across all rotation angles. This work enables accurate characterization of distributed disorder from the fabrication process and facilitates calibration-free imaging in multimode fibers.

Published
2024
Full Text
View/download PDF

4. Shaping single photons through multimode optical fibers using mechanical perturbations

Author

Ronen Shekel, Ohad Lib, Rodrigo Gutiérrez-Cuevas, Sébastien M. Popoff, Alexander Ling, and Yaron Bromberg

Subjects
Applied optics. Photonics, TA1501-1820
Abstract

Multimode optical fibers support low-loss transmission of multiple spatial modes, allowing for the transport of high-dimensional, spatially encoded information. In particular, encoding quantum information in the transverse shape of photons may boost the capacity of quantum channels while using existing infrastructure. However, when photons propagate through a multimode fiber, their transverse shape gets scrambled because of mode mixing and modal interference. This is usually corrected using free-space spatial light modulators, inhibiting a robust all-fiber operation. In this work, we demonstrate an all-fiber approach for controlling the shape of single photons and the spatial correlations between entangled photon pairs, using carefully controlled mechanical perturbations of the fiber. We optimize these perturbations to localize the spatial distribution of a single photon or the spatial correlations of photon pairs in a single spot, enhancing the signal in the optimized spot by over an order of magnitude. Using the same approach, we show a similar enhancement for coupling light from a multimode fiber into a single-mode fiber.

Published
2023
Full Text
View/download PDF

5. A practical guide to digital micro-mirror devices (DMDs) for wavefront shaping

Author

Sébastien M Popoff, Rodrigo Gutiérrez-Cuevas, Yaron Bromberg, and Maxime W Matthés

Subjects
guide, digital micro-mirroring devices, wavefront shaping, SLM, tutorial, Applied optics. Photonics, TA1501-1820, Optics. Light, QC350-467
Abstract

Digital micromirror devices have gained popularity in wavefront shaping, offering a high frame rate alternative to liquid crystal spatial light modulators. They are relatively inexpensive, offer high resolution, are easy to operate, and a single device can be used in a broad optical bandwidth. However, some technical drawbacks must be considered to achieve optimal performance. These issues, often undocumented by manufacturers, mostly stem from the device’s original design for video projection applications. Herein, we present a guide to characterize and mitigate these effects. Our focus is on providing simple and practical solutions that can be easily incorporated into a typical wavefront shaping setup.

Published
2024
Full Text
View/download PDF

6. Tutorial: How to build and control an all-fiber wavefront modulator using mechanical perturbations

Author

Ronen Shekel, Kfir Sulimany, Shachar Resisi, Zohar Finkelstein, Ohad Lib, Sébastien M Popoff, and Yaron Bromberg

Subjects
multimode fibers, wavefront shaping, fiber piano, Applied optics. Photonics, TA1501-1820, Optics. Light, QC350-467
Abstract

Multimode optical fibers support the dense, low-loss transmission of many spatial modes, making them attractive for technologies such as communications and imaging. However, information propagating through multimode fibers is scrambled, due to modal dispersion and mode mixing. This is usually rectified using wavefront shaping techniques with devices such as spatial light modulators. Recently, we demonstrated an all-fiber system for controlling light propagation inside multimode fibers using mechanical perturbations, called the fiber piano. In this tutorial we explain the design considerations and experimental methods needed to build a fiber piano, and review applications where fiber pianos have been used.

Published
2024
Full Text
View/download PDF

7. Gemma 2: Improving Open Language Models at a Practical Size

Author

Gemma Team, Riviere, Morgane, Pathak, Shreya, Sessa, Pier Giuseppe, Hardin, Cassidy, Bhupatiraju, Surya, Hussenot, Léonard, Mesnard, Thomas, Shahriari, Bobak, Ramé, Alexandre, Ferret, Johan, Liu, Peter, Tafti, Pouya, Friesen, Abe, Casbon, Michelle, Ramos, Sabela, Kumar, Ravin, Lan, Charline Le, Jerome, Sammy, Tsitsulin, Anton, Vieillard, Nino, Stanczyk, Piotr, Girgin, Sertan, Momchev, Nikola, Hoffman, Matt, Thakoor, Shantanu, Grill, Jean-Bastien, Neyshabur, Behnam, Bachem, Olivier, Walton, Alanna, Severyn, Aliaksei, Parrish, Alicia, Ahmad, Aliya, Hutchison, Allen, Abdagic, Alvin, Carl, Amanda, Shen, Amy, Brock, Andy, Coenen, Andy, Laforge, Anthony, Paterson, Antonia, Bastian, Ben, Piot, Bilal, Wu, Bo, Royal, Brandon, Chen, Charlie, Kumar, Chintu, Perry, Chris, Welty, Chris, Choquette-Choo, Christopher A., Sinopalnikov, Danila, Weinberger, David, Vijaykumar, Dimple, Rogozińska, Dominika, Herbison, Dustin, Bandy, Elisa, Wang, Emma, Noland, Eric, Moreira, Erica, Senter, Evan, Eltyshev, Evgenii, Visin, Francesco, Rasskin, Gabriel, Wei, Gary, Cameron, Glenn, Martins, Gus, Hashemi, Hadi, Klimczak-Plucińska, Hanna, Batra, Harleen, Dhand, Harsh, Nardini, Ivan, Mein, Jacinda, Zhou, Jack, Svensson, James, Stanway, Jeff, Chan, Jetha, Zhou, Jin Peng, Carrasqueira, Joana, Iljazi, Joana, Becker, Jocelyn, Fernandez, Joe, van Amersfoort, Joost, Gordon, Josh, Lipschultz, Josh, Newlan, Josh, Ji, Ju-yeong, Mohamed, Kareem, Badola, Kartikeya, Black, Kat, Millican, Katie, McDonell, Keelin, Nguyen, Kelvin, Sodhia, Kiranbir, Greene, Kish, Sjoesund, Lars Lowe, Usui, Lauren, Sifre, Laurent, Heuermann, Lena, Lago, Leticia, McNealus, Lilly, Soares, Livio Baldini, Kilpatrick, Logan, Dixon, Lucas, Martins, Luciano, Reid, Machel, Singh, Manvinder, Iverson, Mark, Görner, Martin, Velloso, Mat, Wirth, Mateo, Davidow, Matt, Miller, Matt, Rahtz, Matthew, Watson, Matthew, Risdal, Meg, Kazemi, Mehran, Moynihan, Michael, Zhang, Ming, Kahng, Minsuk, Park, Minwoo, Rahman, Mofi, Khatwani, Mohit, Dao, Natalie, Bardoliwalla, Nenshad, Devanathan, Nesh, Dumai, Neta, Chauhan, Nilay, Wahltinez, Oscar, Botarda, Pankil, Barnes, Parker, Barham, Paul, Michel, Paul, Jin, Pengchong, Georgiev, Petko, Culliton, Phil, Kuppala, Pradeep, Comanescu, Ramona, Merhej, Ramona, Jana, Reena, Rokni, Reza Ardeshir, Agarwal, Rishabh, Mullins, Ryan, Saadat, Samaneh, Carthy, Sara Mc, Cogan, Sarah, Perrin, Sarah, Arnold, Sébastien M. R., Krause, Sebastian, Dai, Shengyang, Garg, Shruti, Sheth, Shruti, Ronstrom, Sue, Chan, Susan, Jordan, Timothy, Yu, Ting, Eccles, Tom, Hennigan, Tom, Kocisky, Tomas, Doshi, Tulsee, Jain, Vihan, Yadav, Vikas, Meshram, Vilobh, Dharmadhikari, Vishal, Barkley, Warren, Wei, Wei, Ye, Wenming, Han, Woohyun, Kwon, Woosuk, Xu, Xiang, Shen, Zhe, Gong, Zhitao, Wei, Zichuan, Cotruta, Victor, Kirk, Phoebe, Rao, Anand, Giang, Minh, Peran, Ludovic, Warkentin, Tris, Collins, Eli, Barral, Joelle, Ghahramani, Zoubin, Hadsell, Raia, Sculley, D., Banks, Jeanine, Dragan, Anca, Petrov, Slav, Vinyals, Oriol, Dean, Jeff, Hassabis, Demis, Kavukcuoglu, Koray, Farabet, Clement, Buchatskaya, Elena, Borgeaud, Sebastian, Fiedel, Noah, Joulin, Armand, Kenealy, Kathleen, Dadashi, Robert, and Andreev, Alek

Subjects
Computer Science - Computation and Language, Computer Science - Artificial Intelligence
Abstract

In this work, we introduce Gemma 2, a new addition to the Gemma family of lightweight, state-of-the-art open models, ranging in scale from 2 billion to 27 billion parameters. In this new version, we apply several known technical modifications to the Transformer architecture, such as interleaving local-global attentions (Beltagy et al., 2020a) and group-query attention (Ainslie et al., 2023). We also train the 2B and 9B models with knowledge distillation (Hinton et al., 2015) instead of next token prediction. The resulting models deliver the best performance for their size, and even offer competitive alternatives to models that are 2-3 times bigger. We release all our models to the community.

Published
2024

8. Can Long-Context Language Models Subsume Retrieval, RAG, SQL, and More?

Author

Lee, Jinhyuk, Chen, Anthony, Dai, Zhuyun, Dua, Dheeru, Sachan, Devendra Singh, Boratko, Michael, Luan, Yi, Arnold, Sébastien M. R., Perot, Vincent, Dalmia, Siddharth, Hu, Hexiang, Lin, Xudong, Pasupat, Panupong, Amini, Aida, Cole, Jeremy R., Riedel, Sebastian, Naim, Iftekhar, Chang, Ming-Wei, and Guu, Kelvin

Subjects
Computer Science - Computation and Language, Computer Science - Artificial Intelligence, Computer Science - Information Retrieval
Abstract

Long-context language models (LCLMs) have the potential to revolutionize our approach to tasks traditionally reliant on external tools like retrieval systems or databases. Leveraging LCLMs' ability to natively ingest and process entire corpora of information offers numerous advantages. It enhances user-friendliness by eliminating the need for specialized knowledge of tools, provides robust end-to-end modeling that minimizes cascading errors in complex pipelines, and allows for the application of sophisticated prompting techniques across the entire system. To assess this paradigm shift, we introduce LOFT, a benchmark of real-world tasks requiring context up to millions of tokens designed to evaluate LCLMs' performance on in-context retrieval and reasoning. Our findings reveal LCLMs' surprising ability to rival state-of-the-art retrieval and RAG systems, despite never having been explicitly trained for these tasks. However, LCLMs still face challenges in areas like compositional reasoning that are required in SQL-like tasks. Notably, prompting strategies significantly influence performance, emphasizing the need for continued research as context lengths grow. Overall, LOFT provides a rigorous testing ground for LCLMs, showcasing their potential to supplant existing paradigms and tackle novel tasks as model capabilities scale., Comment: 29 pages. Dataset available at https://github.com/google-deepmind/loft

Published
2024

9. Inferring Size-Based Functional Responses From the Physical Properties of the Medium

Author

Sébastien M. J. Portalier, Gregor F. Fussmann, Michel Loreau, and Mehdi Cherif

Subjects
functional response, predator, prey, medium, body size, mechanics, Evolution, QH359-425, Ecology, QH540-549.5
Abstract

First derivations of the functional response were mechanistic, but subsequent uses of these functions tended to be phenomenological. Further understanding of the mechanisms underpinning predator-prey relationships might lead to novel insights into functional response in natural systems. Because recent consideration of the physical properties of the environment has improved our understanding of predator-prey interactions, we advocate the use of physics-based approaches for the derivation of the functional response from first principles. These physical factors affect the functional response by constraining the ability of both predators and prey to move according to their size. A physics-based derivation of the functional response should thus consider the movement of organisms in relation to their physical environment. One recent article presents a model along these criteria. As an initial validation of our claim, we use a slightly modified version of this model to derive the classical parameters of the functional response (i.e., attack rate and handling time) of aquatic organisms, as affected by body size, buoyancy, water density and viscosity. We compared the predictions to relevant data. Our model provided good fit for most parameters, but failed to predict handling time. Remarkably, this is the only parameter whose derivation did not rely on physical principles. Parameters in the model were not estimated from observational data. Hence, systematic discrepancies between predictions and real data point immediately to errors in the model. An added benefit to functional response derivation from physical principles is thus to provide easy ways to validate or falsify hypotheses about predator-prey relationships.

Published
2022
Full Text
View/download PDF

10. The Hepatokine TSK does not affect brown fat thermogenic capacity, body weight gain, and glucose homeostasis

Author

Mathilde Mouchiroud, Étienne Camiré, Manal Aldow, Alexandre Caron, Éric Jubinville, Laurie Turcotte, Inés Kaci, Marie-Josée Beaulieu, Christian Roy, Sébastien M. Labbé, Thibault V. Varin, Yves Gélinas, Jennifer Lamothe, Jocelyn Trottier, Patricia L. Mitchell, Frédéric Guénard, William T. Festuccia, Philippe Joubert, Christopher F. Rose, Constantine J. Karvellas, Olivier Barbier, Mathieu C. Morissette, André Marette, and Mathieu Laplante

Subjects
Internal medicine, RC31-1245
Abstract

Objectives: Hepatokines are proteins secreted by the liver that impact the functions of the liver and various tissues through autocrine, paracrine, and endocrine signaling. Recently, Tsukushi (TSK) was identified as a new hepatokine that is induced by obesity and cold exposure. It was proposed that TSK controls sympathetic innervation and thermogenesis in brown adipose tissue (BAT) and that loss of TSK protects against diet-induced obesity and improves glucose homeostasis. Here we report the impact of deleting and/or overexpressing TSK on BAT thermogenic capacity, body weight regulation, and glucose homeostasis. Methods: We measured the expression of thermogenic genes and markers of BAT innervation and activation in TSK-null and TSK-overexpressing mice. Body weight, body temperature, and parameters of glucose homeostasis were also assessed in the context of TSK loss and overexpression. Results: The loss of TSK did not affect the thermogenic activation of BAT. We found that TSK-null mice were not protected against the development of obesity and did not show improvement in glucose tolerance. The overexpression of TSK also failed to modulate thermogenesis, body weight gain, and glucose homeostasis in mice. Conclusions: TSK is not a significant regulator of BAT thermogenesis and is unlikely to represent an effective target to prevent obesity and improve glucose homeostasis. Keywords: Tsukushi, Hepatokine, Brown adipose tissue, Thermogenesis, Glucose homeostasis, Obesity

Published
2019
Full Text
View/download PDF

11. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context

Author

Gemini Team, Georgiev, Petko, Lei, Ving Ian, Burnell, Ryan, Bai, Libin, Gulati, Anmol, Tanzer, Garrett, Vincent, Damien, Pan, Zhufeng, Wang, Shibo, Mariooryad, Soroosh, Ding, Yifan, Geng, Xinyang, Alcober, Fred, Frostig, Roy, Omernick, Mark, Walker, Lexi, Paduraru, Cosmin, Sorokin, Christina, Tacchetti, Andrea, Gaffney, Colin, Daruki, Samira, Sercinoglu, Olcan, Gleicher, Zach, Love, Juliette, Voigtlaender, Paul, Jain, Rohan, Surita, Gabriela, Mohamed, Kareem, Blevins, Rory, Ahn, Junwhan, Zhu, Tao, Kawintiranon, Kornraphop, Firat, Orhan, Gu, Yiming, Zhang, Yujing, Rahtz, Matthew, Faruqui, Manaal, Clay, Natalie, Gilmer, Justin, Co-Reyes, JD, Penchev, Ivo, Zhu, Rui, Morioka, Nobuyuki, Hui, Kevin, Haridasan, Krishna, Campos, Victor, Mahdieh, Mahdis, Guo, Mandy, Hassan, Samer, Kilgour, Kevin, Vezer, Arpi, Cheng, Heng-Tze, de Liedekerke, Raoul, Goyal, Siddharth, Barham, Paul, Strouse, DJ, Noury, Seb, Adler, Jonas, Sundararajan, Mukund, Vikram, Sharad, Lepikhin, Dmitry, Paganini, Michela, Garcia, Xavier, Yang, Fan, Valter, Dasha, Trebacz, Maja, Vodrahalli, Kiran, Asawaroengchai, Chulayuth, Ring, Roman, Kalb, Norbert, Soares, Livio Baldini, Brahma, Siddhartha, Steiner, David, Yu, Tianhe, Mentzer, Fabian, He, Antoine, Gonzalez, Lucas, Xu, Bibo, Kaufman, Raphael Lopez, Shafey, Laurent El, Oh, Junhyuk, Hennigan, Tom, Driessche, George van den, Odoom, Seth, Lucic, Mario, Roelofs, Becca, Lall, Sid, Marathe, Amit, Chan, Betty, Ontanon, Santiago, He, Luheng, Teplyashin, Denis, Lai, Jonathan, Crone, Phil, Damoc, Bogdan, Ho, Lewis, Riedel, Sebastian, Lenc, Karel, Yeh, Chih-Kuan, Chowdhery, Aakanksha, Xu, Yang, Kazemi, Mehran, Amid, Ehsan, Petrushkina, Anastasia, Swersky, Kevin, Khodaei, Ali, Chen, Gowoon, Larkin, Chris, Pinto, Mario, Yan, Geng, Badia, Adria Puigdomenech, Patil, Piyush, Hansen, Steven, Orr, Dave, Arnold, Sebastien M. R., Grimstad, Jordan, Dai, Andrew, Douglas, Sholto, Sinha, Rishika, Yadav, Vikas, Chen, Xi, Gribovskaya, Elena, Austin, Jacob, Zhao, Jeffrey, Patel, Kaushal, Komarek, Paul, Austin, Sophia, Borgeaud, Sebastian, Friso, Linda, Goyal, Abhimanyu, Caine, Ben, Cao, Kris, Chung, Da-Woon, Lamm, Matthew, Barth-Maron, Gabe, Kagohara, Thais, Olszewska, Kate, Chen, Mia, Shivakumar, Kaushik, Agarwal, Rishabh, Godhia, Harshal, Rajwar, Ravi, Snaider, Javier, Dotiwalla, Xerxes, Liu, Yuan, Barua, Aditya, Ungureanu, Victor, Zhang, Yuan, Batsaikhan, Bat-Orgil, Wirth, Mateo, Qin, James, Danihelka, Ivo, Doshi, Tulsee, Chadwick, Martin, Chen, Jilin, Jain, Sanil, Le, Quoc, Kar, Arjun, Gurumurthy, Madhu, Li, Cheng, Sang, Ruoxin, Liu, Fangyu, Lamprou, Lampros, Munoz, Rich, Lintz, Nathan, Mehta, Harsh, Howard, Heidi, Reynolds, Malcolm, Aroyo, Lora, Wang, Quan, Blanco, Lorenzo, Cassirer, Albin, Griffith, Jordan, Das, Dipanjan, Lee, Stephan, Sygnowski, Jakub, Fisher, Zach, Besley, James, Powell, Richard, Ahmed, Zafarali, Paulus, Dominik, Reitter, David, Borsos, Zalan, Joshi, Rishabh, Pope, Aedan, Hand, Steven, Selo, Vittorio, Jain, Vihan, Sethi, Nikhil, Goel, Megha, Makino, Takaki, May, Rhys, Yang, Zhen, Schalkwyk, Johan, Butterfield, Christina, Hauth, Anja, Goldin, Alex, Hawkins, Will, Senter, Evan, Brin, Sergey, Woodman, Oliver, Ritter, Marvin, Noland, Eric, Giang, Minh, Bolina, Vijay, Lee, Lisa, Blyth, Tim, Mackinnon, Ian, Reid, Machel, Sarvana, Obaid, Silver, David, Chen, Alexander, Wang, Lily, Maggiore, Loren, Chang, Oscar, Attaluri, Nithya, Thornton, Gregory, Chiu, Chung-Cheng, Bunyan, Oskar, Levine, Nir, Chung, Timothy, Eltyshev, Evgenii, Si, Xiance, Lillicrap, Timothy, Brady, Demetra, Aggarwal, Vaibhav, Wu, Boxi, Xu, Yuanzhong, McIlroy, Ross, Badola, Kartikeya, Sandhu, Paramjit, Moreira, Erica, Stokowiec, Wojciech, Hemsley, Ross, Li, Dong, Tudor, Alex, Shyam, Pranav, Rahimtoroghi, Elahe, Haykal, Salem, Sprechmann, Pablo, Zhou, Xiang, Mincu, Diana, Li, Yujia, Addanki, Ravi, Krishna, Kalpesh, Wu, Xiao, Frechette, Alexandre, Eyal, Matan, Dafoe, Allan, Lacey, Dave, Whang, Jay, Avrahami, Thi, Zhang, Ye, Taropa, Emanuel, Lin, Hanzhao, Toyama, Daniel, Rutherford, Eliza, Sano, Motoki, Choe, HyunJeong, Tomala, Alex, Safranek-Shrader, Chalence, Kassner, Nora, Pajarskas, Mantas, Harvey, Matt, Sechrist, Sean, Fortunato, Meire, Lyu, Christina, Elsayed, Gamaleldin, Kuang, Chenkai, Lottes, James, Chu, Eric, Jia, Chao, Chen, Chih-Wei, Humphreys, Peter, Baumli, Kate, Tao, Connie, Samuel, Rajkumar, Santos, Cicero Nogueira dos, Andreassen, Anders, Rakićević, Nemanja, Grewe, Dominik, Kumar, Aviral, Winkler, Stephanie, Caton, Jonathan, Brock, Andrew, Dalmia, Sid, Sheahan, Hannah, Barr, Iain, Miao, Yingjie, Natsev, Paul, Devlin, Jacob, Behbahani, Feryal, Prost, Flavien, Sun, Yanhua, Myaskovsky, Artiom, Pillai, Thanumalayan Sankaranarayana, Hurt, Dan, Lazaridou, Angeliki, Xiong, Xi, Zheng, Ce, Pardo, Fabio, Li, Xiaowei, Horgan, Dan, Stanton, Joe, Ambar, Moran, Xia, Fei, Lince, Alejandro, Wang, Mingqiu, Mustafa, Basil, Webson, Albert, Lee, Hyo, Anil, Rohan, Wicke, Martin, Dozat, Timothy, Sinha, Abhishek, Piqueras, Enrique, Dabir, Elahe, Upadhyay, Shyam, Boral, Anudhyan, Hendricks, Lisa Anne, Fry, Corey, Djolonga, Josip, Su, Yi, Walker, Jake, Labanowski, Jane, Huang, Ronny, Misra, Vedant, Chen, Jeremy, Skerry-Ryan, RJ, Singh, Avi, Rijhwani, Shruti, Yu, Dian, Castro-Ros, Alex, Changpinyo, Beer, Datta, Romina, Bagri, Sumit, Hrafnkelsson, Arnar Mar, Maggioni, Marcello, Zheng, Daniel, Sulsky, Yury, Hou, Shaobo, Paine, Tom Le, Yang, Antoine, Riesa, Jason, Rogozinska, Dominika, Marcus, Dror, Badawy, Dalia El, Zhang, Qiao, Wang, Luyu, Miller, Helen, Greer, Jeremy, Sjos, Lars Lowe, Nova, Azade, Zen, Heiga, Chaabouni, Rahma, Rosca, Mihaela, Jiang, Jiepu, Chen, Charlie, Liu, Ruibo, Sainath, Tara, Krikun, Maxim, Polozov, Alex, Lespiau, Jean-Baptiste, Newlan, Josh, Cankara, Zeyncep, Kwak, Soo, Xu, Yunhan, Chen, Phil, Coenen, Andy, Meyer, Clemens, Tsihlas, Katerina, Ma, Ada, Gottweis, Juraj, Xing, Jinwei, Gu, Chenjie, Miao, Jin, Frank, Christian, Cankara, Zeynep, Ganapathy, Sanjay, Dasgupta, Ishita, Hughes-Fitt, Steph, Chen, Heng, Reid, David, Rong, Keran, Fan, Hongmin, van Amersfoort, Joost, Zhuang, Vincent, Cohen, Aaron, Gu, Shixiang Shane, Mohananey, Anhad, Ilic, Anastasija, Tobin, Taylor, Wieting, John, Bortsova, Anna, Thacker, Phoebe, Wang, Emma, Caveness, Emily, Chiu, Justin, Sezener, Eren, Kaskasoli, Alex, Baker, Steven, Millican, Katie, Elhawaty, Mohamed, Aisopos, Kostas, Lebsack, Carl, Byrd, Nathan, Dai, Hanjun, Jia, Wenhao, Wiethoff, Matthew, Davoodi, Elnaz, Weston, Albert, Yagati, Lakshman, Ahuja, Arun, Gao, Isabel, Pundak, Golan, Zhang, Susan, Azzam, Michael, Sim, Khe Chai, Caelles, Sergi, Keeling, James, Sharma, Abhanshu, Swing, Andy, Li, YaGuang, Liu, Chenxi, Bostock, Carrie Grimes, Bansal, Yamini, Nado, Zachary, Anand, Ankesh, Lipschultz, Josh, Karmarkar, Abhijit, Proleev, Lev, Ittycheriah, Abe, Yeganeh, Soheil Hassas, Polovets, George, Faust, Aleksandra, Sun, Jiao, Rrustemi, Alban, Li, Pen, Shivanna, Rakesh, Liu, Jeremiah, Welty, Chris, Lebron, Federico, Baddepudi, Anirudh, Krause, Sebastian, Parisotto, Emilio, Soricut, Radu, Xu, Zheng, Bloxwich, Dawn, Johnson, Melvin, Neyshabur, Behnam, Mao-Jones, Justin, Wang, Renshen, Ramasesh, Vinay, Abbas, Zaheer, Guez, Arthur, Segal, Constant, Nguyen, Duc Dung, Svensson, James, Hou, Le, York, Sarah, Milan, Kieran, Bridgers, Sophie, Gworek, Wiktor, Tagliasacchi, Marco, Lee-Thorp, James, Chang, Michael, Guseynov, Alexey, Hartman, Ale Jakse, Kwong, Michael, Zhao, Ruizhe, Kashem, Sheleem, Cole, Elizabeth, Miech, Antoine, Tanburn, Richard, Phuong, Mary, Pavetic, Filip, Cevey, Sebastien, Comanescu, Ramona, Ives, Richard, Yang, Sherry, Du, Cosmo, Li, Bo, Zhang, Zizhao, Iinuma, Mariko, Hu, Clara Huiyi, Roy, Aurko, Bijwadia, Shaan, Zhu, Zhenkai, Martins, Danilo, Saputro, Rachel, Gergely, Anita, Zheng, Steven, Jia, Dawei, Antonoglou, Ioannis, Sadovsky, Adam, Gu, Shane, Bi, Yingying, Andreev, Alek, Samangooei, Sina, Khan, Mina, Kocisky, Tomas, Filos, Angelos, Kumar, Chintu, Bishop, Colton, Yu, Adams, Hodkinson, Sarah, Mittal, Sid, Shah, Premal, Moufarek, Alexandre, Cheng, Yong, Bloniarz, Adam, Lee, Jaehoon, Pejman, Pedram, Michel, Paul, Spencer, Stephen, Feinberg, Vladimir, Xiong, Xuehan, Savinov, Nikolay, Smith, Charlotte, Shakeri, Siamak, Tran, Dustin, Chesus, Mary, Bohnet, Bernd, Tucker, George, von Glehn, Tamara, Muir, Carrie, Mao, Yiran, Kazawa, Hideto, Slone, Ambrose, Soparkar, Kedar, Shrivastava, Disha, Cobon-Kerr, James, Sharman, Michael, Pavagadhi, Jay, Araya, Carlos, Misiunas, Karolis, Ghelani, Nimesh, Laskin, Michael, Barker, David, Li, Qiujia, Briukhov, Anton, Houlsby, Neil, Glaese, Mia, Lakshminarayanan, Balaji, Schucher, Nathan, Tang, Yunhao, Collins, Eli, Lim, Hyeontaek, Feng, Fangxiaoyu, Recasens, Adria, Lai, Guangda, Magni, Alberto, De Cao, Nicola, Siddhant, Aditya, Ashwood, Zoe, Orbay, Jordi, Dehghani, Mostafa, Brennan, Jenny, He, Yifan, Xu, Kelvin, Gao, Yang, Saroufim, Carl, Molloy, James, Wu, Xinyi, Arnold, Seb, Chang, Solomon, Schrittwieser, Julian, Buchatskaya, Elena, Radpour, Soroush, Polacek, Martin, Giordano, Skye, Bapna, Ankur, Tokumine, Simon, Hellendoorn, Vincent, Sottiaux, Thibault, Cogan, Sarah, Severyn, Aliaksei, Saleh, Mohammad, Thakoor, Shantanu, Shefey, Laurent, Qiao, Siyuan, Gaba, Meenu, Chang, Shuo-yiin, Swanson, Craig, Zhang, Biao, Lee, Benjamin, Rubenstein, Paul Kishan, Song, Gan, Kwiatkowski, Tom, Koop, Anna, Kannan, Ajay, Kao, David, Schuh, Parker, Stjerngren, Axel, Ghiasi, Golnaz, Gibson, Gena, Vilnis, Luke, Yuan, Ye, Ferreira, Felipe Tiengo, Kamath, Aishwarya, Klimenko, Ted, Franko, Ken, Xiao, Kefan, Bhattacharya, Indro, Patel, Miteyan, Wang, Rui, Morris, Alex, Strudel, Robin, Sharma, Vivek, Choy, Peter, Hashemi, Sayed Hadi, Landon, Jessica, Finkelstein, Mara, Jhakra, Priya, Frye, Justin, Barnes, Megan, Mauger, Matthew, Daun, Dennis, Baatarsukh, Khuslen, Tung, Matthew, Farhan, Wael, Michalewski, Henryk, Viola, Fabio, Quitry, Felix de Chaumont, Lan, Charline Le, Hudson, Tom, Wang, Qingze, Fischer, Felix, Zheng, Ivy, White, Elspeth, Dragan, Anca, Alayrac, Jean-baptiste, Ni, Eric, Pritzel, Alexander, Iwanicki, Adam, Isard, Michael, Bulanova, Anna, Zilka, Lukas, Dyer, Ethan, Sachan, Devendra, Srinivasan, Srivatsan, Muckenhirn, Hannah, Cai, Honglong, Mandhane, Amol, Tariq, Mukarram, Rae, Jack W., Wang, Gary, Ayoub, Kareem, FitzGerald, Nicholas, Zhao, Yao, Han, Woohyun, Alberti, Chris, Garrette, Dan, Krishnakumar, Kashyap, Gimenez, Mai, Levskaya, Anselm, Sohn, Daniel, Matak, Josip, Iturrate, Inaki, Chang, Michael B., Xiang, Jackie, Cao, Yuan, Ranka, Nishant, Brown, Geoff, Hutter, Adrian, Mirrokni, Vahab, Chen, Nanxin, Yao, Kaisheng, Egyed, Zoltan, Galilee, Francois, Liechty, Tyler, Kallakuri, Praveen, Palmer, Evan, Ghemawat, Sanjay, Liu, Jasmine, Tao, David, Thornton, Chloe, Green, Tim, Jasarevic, Mimi, Lin, Sharon, Cotruta, Victor, Tan, Yi-Xuan, Fiedel, Noah, Yu, Hongkun, Chi, Ed, Neitz, Alexander, Heitkaemper, Jens, Sinha, Anu, Zhou, Denny, Sun, Yi, Kaed, Charbel, Hulse, Brice, Mishra, Swaroop, Georgaki, Maria, Kudugunta, Sneha, Farabet, Clement, Shafran, Izhak, Vlasic, Daniel, Tsitsulin, Anton, Ananthanarayanan, Rajagopal, Carin, Alen, Su, Guolong, Sun, Pei, V, Shashank, Carvajal, Gabriel, Broder, Josef, Comsa, Iulia, Repina, Alena, Wong, William, Chen, Warren Weilun, Hawkins, Peter, Filonov, Egor, Loher, Lucia, Hirnschall, Christoph, Wang, Weiyi, Ye, Jingchen, Burns, Andrea, Cate, Hardie, Wright, Diana Gage, Piccinini, Federico, Zhang, Lei, Lin, Chu-Cheng, Gog, Ionel, Kulizhskaya, Yana, Sreevatsa, Ashwin, Song, Shuang, Cobo, Luis C., Iyer, Anand, Tekur, Chetan, Garrido, Guillermo, Xiao, Zhuyun, Kemp, Rupert, Zheng, Huaixiu Steven, Li, Hui, Agarwal, Ananth, Ngani, Christel, Goshvadi, Kati, Santamaria-Fernandez, Rebeca, Fica, Wojciech, Chen, Xinyun, Gorgolewski, Chris, Sun, Sean, Garg, Roopal, Ye, Xinyu, Eslami, S. M. Ali, Hua, Nan, Simon, Jon, Joshi, Pratik, Kim, Yelin, Tenney, Ian, Potluri, Sahitya, Thiet, Lam Nguyen, Yuan, Quan, Luisier, Florian, Chronopoulou, Alexandra, Scellato, Salvatore, Srinivasan, Praveen, Chen, Minmin, Koverkathu, Vinod, Dalibard, Valentin, Xu, Yaming, Saeta, Brennan, Anderson, Keith, Sellam, Thibault, Fernando, Nick, Huot, Fantine, Jung, Junehyuk, Varadarajan, Mani, Quinn, Michael, Raul, Amit, Le, Maigo, Habalov, Ruslan, Clark, Jon, Jalan, Komal, Bullard, Kalesha, Singhal, Achintya, Luong, Thang, Wang, Boyu, Rajayogam, Sujeevan, Eisenschlos, Julian, Jia, Johnson, Finchelstein, Daniel, Yakubovich, Alex, Balle, Daniel, Fink, Michael, Agarwal, Sameer, Li, Jing, Dvijotham, Dj, Pal, Shalini, Kang, Kai, Konzelmann, Jaclyn, Beattie, Jennifer, Dousse, Olivier, Wu, Diane, Crocker, Remi, Elkind, Chen, Jonnalagadda, Siddhartha Reddy, Lee, Jong, Holtmann-Rice, Dan, Kallarackal, Krystal, Liu, Rosanne, Vnukov, Denis, Vats, Neera, Invernizzi, Luca, Jafari, Mohsen, Zhou, Huanjie, Taylor, Lilly, Prendki, Jennifer, Wu, Marcus, Eccles, Tom, Liu, Tianqi, Kopparapu, Kavya, Beaufays, Francoise, Angermueller, Christof, Marzoca, Andreea, Sarcar, Shourya, Dib, Hilal, Stanway, Jeff, Perbet, Frank, Trdin, Nejc, Sterneck, Rachel, Khorlin, Andrey, Li, Dinghua, Wu, Xihui, Goenka, Sonam, Madras, David, Goldshtein, Sasha, Gierke, Willi, Zhou, Tong, Liu, Yaxin, Liang, Yannie, White, Anais, Li, Yunjie, Singh, Shreya, Bahargam, Sanaz, Epstein, Mark, Basu, Sujoy, Lao, Li, Ozturel, Adnan, Crous, Carl, Zhai, Alex, Lu, Han, Tung, Zora, Gaur, Neeraj, Walton, Alanna, Dixon, Lucas, Zhang, Ming, Globerson, Amir, Uy, Grant, Bolt, Andrew, Wiles, Olivia, Nasr, Milad, Shumailov, Ilia, Selvi, Marco, Piccinno, Francesco, Aguilar, Ricardo, McCarthy, Sara, Khalman, Misha, Shukla, Mrinal, Galic, Vlado, Carpenter, John, Villela, Kevin, Zhang, Haibin, Richardson, Harry, Martens, James, Bosnjak, Matko, Belle, Shreyas Rammohan, Seibert, Jeff, Alnahlawi, Mahmoud, McWilliams, Brian, Singh, Sankalp, Louis, Annie, Ding, Wen, Popovici, Dan, Simicich, Lenin, Knight, Laura, Mehta, Pulkit, Gupta, Nishesh, Shi, Chongyang, Fatehi, Saaber, Mitrovic, Jovana, Grills, Alex, Pagadora, Joseph, Petrova, Dessie, Eisenbud, Danielle, Zhang, Zhishuai, Yates, Damion, Mittal, Bhavishya, Tripuraneni, Nilesh, Assael, Yannis, Brovelli, Thomas, Jain, Prateek, Velimirovic, Mihajlo, Akbulut, Canfer, Mu, Jiaqi, Macherey, Wolfgang, Kumar, Ravin, Xu, Jun, Qureshi, Haroon, Comanici, Gheorghe, Wiesner, Jeremy, Gong, Zhitao, Ruddock, Anton, Bauer, Matthias, Felt, Nick, GP, Anirudh, Arnab, Anurag, Zelle, Dustin, Rothfuss, Jonas, Rosgen, Bill, Shenoy, Ashish, Seybold, Bryan, Li, Xinjian, Mudigonda, Jayaram, Erdogan, Goker, Xia, Jiawei, Simsa, Jiri, Michi, Andrea, Yao, Yi, Yew, Christopher, Kan, Steven, Caswell, Isaac, Radebaugh, Carey, Elisseeff, Andre, Valenzuela, Pedro, McKinney, Kay, Paterson, Kim, Cui, Albert, Latorre-Chimoto, Eri, Kim, Solomon, Zeng, William, Durden, Ken, Ponnapalli, Priya, Sosea, Tiberiu, Choquette-Choo, Christopher A., Manyika, James, Robenek, Brona, Vashisht, Harsha, Pereira, Sebastien, Lam, Hoi, Velic, Marko, Owusu-Afriyie, Denese, Lee, Katherine, Bolukbasi, Tolga, Parrish, Alicia, Lu, Shawn, Park, Jane, Venkatraman, Balaji, Talbert, Alice, Rosique, Lambert, Cheng, Yuchung, Sozanschi, Andrei, Paszke, Adam, Kumar, Praveen, Austin, Jessica, Li, Lu, Salama, Khalid, Kim, Wooyeol, Dukkipati, Nandita, Baryshnikov, Anthony, Kaplanis, Christos, Sheng, XiangHai, Chervonyi, Yuri, Unlu, Caglar, Casas, Diego de Las, Askham, Harry, Tunyasuvunakool, Kathryn, Gimeno, Felix, Poder, Siim, Kwak, Chester, Miecnikowski, Matt, Dimitriev, Alek, Parisi, Aaron, Liu, Dangyi, Tsai, Tomy, Shevlane, Toby, Kouridi, Christina, Garmon, Drew, Goedeckemeyer, Adrian, Brown, Adam R., Vijayakumar, Anitha, Elqursh, Ali, Jazayeri, Sadegh, Huang, Jin, Carthy, Sara Mc, Hoover, Jay, Kim, Lucy, Kumar, Sandeep, Chen, Wei, Biles, Courtney, Bingham, Garrett, Rosen, Evan, Wang, Lisa, Tan, Qijun, Engel, David, Pongetti, Francesco, de Cesare, Dario, Hwang, Dongseong, Yu, Lily, Pullman, Jennifer, Narayanan, Srini, Levin, Kyle, Gopal, Siddharth, Li, Megan, Aharoni, Asaf, Trinh, Trieu, Lo, Jessica, Casagrande, Norman, Vij, Roopali, Matthey, Loic, Ramadhana, Bramandia, Matthews, Austin, Carey, CJ, Johnson, Matthew, Goranova, Kremena, Shah, Rohin, Ashraf, Shereen, Dasgupta, Kingshuk, Larsen, Rasmus, Wang, Yicheng, Vuyyuru, Manish Reddy, Jiang, Chong, Ijazi, Joana, Osawa, Kazuki, Smith, Celine, Boppana, Ramya Sree, Bilal, Taylan, Koizumi, Yuma, Xu, Ying, Altun, Yasemin, Shabat, Nir, Bariach, Ben, Korchemniy, Alex, Choo, Kiam, Ronneberger, Olaf, Iwuanyanwu, Chimezie, Zhao, Shubin, Soergel, David, Hsieh, Cho-Jui, Cai, Irene, Iqbal, Shariq, Sundermeyer, Martin, Chen, Zhe, Bursztein, Elie, Malaviya, Chaitanya, Biadsy, Fadi, Shroff, Prakash, Dhillon, Inderjit, Latkar, Tejasi, Dyer, Chris, Forbes, Hannah, Nicosia, Massimo, Nikolaev, Vitaly, Greene, Somer, Georgiev, Marin, Wang, Pidong, Martin, Nina, Sedghi, Hanie, Zhang, John, Banzal, Praseem, Fritz, Doug, Rao, Vikram, Wang, Xuezhi, Zhang, Jiageng, Patraucean, Viorica, Du, Dayou, Mordatch, Igor, Jurin, Ivan, Liu, Lewis, Dubey, Ayush, Mohan, Abhi, Nowakowski, Janek, Ion, Vlad-Doru, Wei, Nan, Tojo, Reiko, Raad, Maria Abi, Hudson, Drew A., Keshava, Vaishakh, Agrawal, Shubham, Ramirez, Kevin, Wu, Zhichun, Nguyen, Hoang, Liu, Ji, Sewak, Madhavi, Petrini, Bryce, Choi, DongHyun, Philips, Ivan, Wang, Ziyue, Bica, Ioana, Garg, Ankush, Wilkiewicz, Jarek, Agrawal, Priyanka, Guo, Danhao, Xue, Emily, Shaik, Naseer, Leach, Andrew, Khan, Sadh MNM, Wiesinger, Julia, Jerome, Sammy, Chakladar, Abhishek, Wang, Alek Wenjiao, Ornduff, Tina, Abu, Folake, Ghaffarkhah, Alireza, Wainwright, Marcus, Cortes, Mario, Liu, Frederick, Maynez, Joshua, Terzis, Andreas, Samangouei, Pouya, Mansour, Riham, Kępa, Tomasz, Aubet, François-Xavier, Algymr, Anton, Banica, Dan, Weisz, Agoston, Orban, Andras, Senges, Alexandre, Andrejczuk, Ewa, Geller, Mark, Santo, Niccolo Dal, Anklin, Valentin, Merey, Majd Al, Baeuml, Martin, Strohman, Trevor, Bai, Junwen, Petrov, Slav, Wu, Yonghui, Hassabis, Demis, Kavukcuoglu, Koray, Dean, Jeffrey, and Vinyals, Oriol

Subjects
Computer Science - Computation and Language, Computer Science - Artificial Intelligence
Abstract

In this report, we introduce the Gemini 1.5 family of models, representing the next generation of highly compute-efficient multimodal models capable of recalling and reasoning over fine-grained information from millions of tokens of context, including multiple long documents and hours of video and audio. The family includes two new models: (1) an updated Gemini 1.5 Pro, which exceeds the February version on the great majority of capabilities and benchmarks; (2) Gemini 1.5 Flash, a more lightweight variant designed for efficiency with minimal regression in quality. Gemini 1.5 models achieve near-perfect recall on long-context retrieval tasks across modalities, improve the state-of-the-art in long-document QA, long-video QA and long-context ASR, and match or surpass Gemini 1.0 Ultra's state-of-the-art performance across a broad set of benchmarks. Studying the limits of Gemini 1.5's long-context ability, we find continued improvement in next-token prediction and near-perfect retrieval (>99%) up to at least 10M tokens, a generational leap over existing models such as Claude 3.0 (200k) and GPT-4 Turbo (128k). Finally, we highlight real-world use cases, such as Gemini 1.5 collaborating with professionals on completing their tasks achieving 26 to 75% time savings across 10 different job categories, as well as surprising new capabilities of large language models at the frontier; when given a grammar manual for Kalamang, a language with fewer than 200 speakers worldwide, the model learns to translate English to Kalamang at a similar level to a person who learned from the same content.

Published
2024

12. Rhythmic fluctuations of saccadic reaction time arising from visual competition

Author

Samson Chota, Canhuang Luo, Sébastien M. Crouzet, Léa Boyer, Ricardo Kienitz, Michael Christoph Schmid, and Rufin VanRullen

Subjects
Saccadic Reaction Times, Rhythmic Fluctuations, Theta Range, Firing Rate Oscillations, Attentional Sampling, Medicine, Science
Abstract

Abstract Recent research indicates that attentional stimulus selection could be a rhythmic process. In monkey, neurons in V4 and IT exhibit rhythmic spiking activity in the theta range in response to a stimulus. When two stimuli are presented together, the rhythmic neuronal responses to each occur in anti-phase, a result indicative of competitive interactions. In addition, it was recently demonstrated that these alternating oscillations in monkey V4 modulate the speed of saccadic responses to a target flashed on one of the two competing stimuli. Here, we replicate a similar behavioral task in humans (7 participants, each performed 4000 trials) and report a pattern of results consistent with the monkey findings: saccadic response times fluctuate in the theta range (6 Hz), with opposite phase for targets flashed on distinct competing stimuli.

Published
2018
Full Text
View/download PDF

13. Can an LLM-Powered Socially Assistive Robot Effectively and Safely Deliver Cognitive Behavioral Therapy? A Study With University Students

Author

Kian, Mina J., Zong, Mingyu, Fischer, Katrin, Singh, Abhyuday, Velentza, Anna-Maria, Sang, Pau, Upadhyay, Shriya, Gupta, Anika, Faruki, Misha A., Browning, Wallace, Arnold, Sebastien M. R., Krishnamachari, Bhaskar, and Mataric, Maja J.

Subjects
Computer Science - Robotics
Abstract

Cognitive behavioral therapy (CBT) is a widely used therapeutic method for guiding individuals toward restructuring their thinking patterns as a means of addressing anxiety, depression, and other challenges. We developed a large language model (LLM)-powered prompt-engineered socially assistive robot (SAR) that guides participants through interactive CBT at-home exercises. We evaluated the performance of the SAR through a 15-day study with 38 university students randomly assigned to interact daily with the robot or a chatbot (using the same LLM), or complete traditional CBT worksheets throughout the duration of the study. We measured weekly therapeutic outcomes, changes in pre-/post-session anxiety measures, and adherence to completing CBT exercises. We found that self-reported measures of general psychological distress significantly decreased over the study period in the robot and worksheet conditions but not the chatbot condition. Furthermore, the SAR enabled significant single-session improvements for more sessions than the other two conditions combined. Our findings suggest that SAR-guided LLM-powered CBT may be as effective as traditional worksheet methods in supporting therapeutic progress from the beginning to the end of the study and superior in decreasing user anxiety immediately after completing the CBT exercise.

Published
2024

14. Gemini: A Family of Highly Capable Multimodal Models

Author

Gemini Team, Anil, Rohan, Borgeaud, Sebastian, Alayrac, Jean-Baptiste, Yu, Jiahui, Soricut, Radu, Schalkwyk, Johan, Dai, Andrew M., Hauth, Anja, Millican, Katie, Silver, David, Johnson, Melvin, Antonoglou, Ioannis, Schrittwieser, Julian, Glaese, Amelia, Chen, Jilin, Pitler, Emily, Lillicrap, Timothy, Lazaridou, Angeliki, Firat, Orhan, Molloy, James, Isard, Michael, Barham, Paul R., Hennigan, Tom, Lee, Benjamin, Viola, Fabio, Reynolds, Malcolm, Xu, Yuanzhong, Doherty, Ryan, Collins, Eli, Meyer, Clemens, Rutherford, Eliza, Moreira, Erica, Ayoub, Kareem, Goel, Megha, Krawczyk, Jack, Du, Cosmo, Chi, Ed, Cheng, Heng-Tze, Ni, Eric, Shah, Purvi, Kane, Patrick, Chan, Betty, Faruqui, Manaal, Severyn, Aliaksei, Lin, Hanzhao, Li, YaGuang, Cheng, Yong, Ittycheriah, Abe, Mahdieh, Mahdis, Chen, Mia, Sun, Pei, Tran, Dustin, Bagri, Sumit, Lakshminarayanan, Balaji, Liu, Jeremiah, Orban, Andras, Güra, Fabian, Zhou, Hao, Song, Xinying, Boffy, Aurelien, Ganapathy, Harish, Zheng, Steven, Choe, HyunJeong, Weisz, Ágoston, Zhu, Tao, Lu, Yifeng, Gopal, Siddharth, Kahn, Jarrod, Kula, Maciej, Pitman, Jeff, Shah, Rushin, Taropa, Emanuel, Merey, Majd Al, Baeuml, Martin, Chen, Zhifeng, Shafey, Laurent El, Zhang, Yujing, Sercinoglu, Olcan, Tucker, George, Piqueras, Enrique, Krikun, Maxim, Barr, Iain, Savinov, Nikolay, Danihelka, Ivo, Roelofs, Becca, White, Anaïs, Andreassen, Anders, von Glehn, Tamara, Yagati, Lakshman, Kazemi, Mehran, Gonzalez, Lucas, Khalman, Misha, Sygnowski, Jakub, Frechette, Alexandre, Smith, Charlotte, Culp, Laura, Proleev, Lev, Luan, Yi, Chen, Xi, Lottes, James, Schucher, Nathan, Lebron, Federico, Rrustemi, Alban, Clay, Natalie, Crone, Phil, Kocisky, Tomas, Zhao, Jeffrey, Perz, Bartek, Yu, Dian, Howard, Heidi, Bloniarz, Adam, Rae, Jack W., Lu, Han, Sifre, Laurent, Maggioni, Marcello, Alcober, Fred, Garrette, Dan, Barnes, Megan, Thakoor, Shantanu, Austin, Jacob, Barth-Maron, Gabriel, Wong, William, Joshi, Rishabh, Chaabouni, Rahma, Fatiha, Deeni, Ahuja, Arun, Tomar, Gaurav Singh, Senter, Evan, Chadwick, Martin, Kornakov, Ilya, Attaluri, Nithya, Iturrate, Iñaki, Liu, Ruibo, Li, Yunxuan, Cogan, Sarah, Chen, Jeremy, Jia, Chao, Gu, Chenjie, Zhang, Qiao, Grimstad, Jordan, Hartman, Ale Jakse, Garcia, Xavier, Pillai, Thanumalayan Sankaranarayana, Devlin, Jacob, Laskin, Michael, Casas, Diego de Las, Valter, Dasha, Tao, Connie, Blanco, Lorenzo, Badia, Adrià Puigdomènech, Reitter, David, Chen, Mianna, Brennan, Jenny, Rivera, Clara, Brin, Sergey, Iqbal, Shariq, Surita, Gabriela, Labanowski, Jane, Rao, Abhi, Winkler, Stephanie, Parisotto, Emilio, Gu, Yiming, Olszewska, Kate, Addanki, Ravi, Miech, Antoine, Louis, Annie, Teplyashin, Denis, Brown, Geoff, Catt, Elliot, Balaguer, Jan, Xiang, Jackie, Wang, Pidong, Ashwood, Zoe, Briukhov, Anton, Webson, Albert, Ganapathy, Sanjay, Sanghavi, Smit, Kannan, Ajay, Chang, Ming-Wei, Stjerngren, Axel, Djolonga, Josip, Sun, Yuting, Bapna, Ankur, Aitchison, Matthew, Pejman, Pedram, Michalewski, Henryk, Yu, Tianhe, Wang, Cindy, Love, Juliette, Ahn, Junwhan, Bloxwich, Dawn, Han, Kehang, Humphreys, Peter, Sellam, Thibault, Bradbury, James, Godbole, Varun, Samangooei, Sina, Damoc, Bogdan, Kaskasoli, Alex, Arnold, Sébastien M. R., Vasudevan, Vijay, Agrawal, Shubham, Riesa, Jason, Lepikhin, Dmitry, Tanburn, Richard, Srinivasan, Srivatsan, Lim, Hyeontaek, Hodkinson, Sarah, Shyam, Pranav, Ferret, Johan, Hand, Steven, Garg, Ankush, Paine, Tom Le, Li, Jian, Li, Yujia, Giang, Minh, Neitz, Alexander, Abbas, Zaheer, York, Sarah, Reid, Machel, Cole, Elizabeth, Chowdhery, Aakanksha, Das, Dipanjan, Rogozińska, Dominika, Nikolaev, Vitaliy, Sprechmann, Pablo, Nado, Zachary, Zilka, Lukas, Prost, Flavien, He, Luheng, Monteiro, Marianne, Mishra, Gaurav, Welty, Chris, Newlan, Josh, Jia, Dawei, Allamanis, Miltiadis, Hu, Clara Huiyi, de Liedekerke, Raoul, Gilmer, Justin, Saroufim, Carl, Rijhwani, Shruti, Hou, Shaobo, Shrivastava, Disha, Baddepudi, Anirudh, Goldin, Alex, Ozturel, Adnan, Cassirer, Albin, Xu, Yunhan, Sohn, Daniel, Sachan, Devendra, Amplayo, Reinald Kim, Swanson, Craig, Petrova, Dessie, Narayan, Shashi, Guez, Arthur, Brahma, Siddhartha, Landon, Jessica, Patel, Miteyan, Zhao, Ruizhe, Villela, Kevin, Wang, Luyu, Jia, Wenhao, Rahtz, Matthew, Giménez, Mai, Yeung, Legg, Keeling, James, Georgiev, Petko, Mincu, Diana, Wu, Boxi, Haykal, Salem, Saputro, Rachel, Vodrahalli, Kiran, Qin, James, Cankara, Zeynep, Sharma, Abhanshu, Fernando, Nick, Hawkins, Will, Neyshabur, Behnam, Kim, Solomon, Hutter, Adrian, Agrawal, Priyanka, Castro-Ros, Alex, Driessche, George van den, Wang, Tao, Yang, Fan, Chang, Shuo-yiin, Komarek, Paul, McIlroy, Ross, Lučić, Mario, Zhang, Guodong, Farhan, Wael, Sharman, Michael, Natsev, Paul, Michel, Paul, Bansal, Yamini, Qiao, Siyuan, Cao, Kris, Shakeri, Siamak, Butterfield, Christina, Chung, Justin, Rubenstein, Paul Kishan, Agrawal, Shivani, Mensch, Arthur, Soparkar, Kedar, Lenc, Karel, Chung, Timothy, Pope, Aedan, Maggiore, Loren, Kay, Jackie, Jhakra, Priya, Wang, Shibo, Maynez, Joshua, Phuong, Mary, Tobin, Taylor, Tacchetti, Andrea, Trebacz, Maja, Robinson, Kevin, Katariya, Yash, Riedel, Sebastian, Bailey, Paige, Xiao, Kefan, Ghelani, Nimesh, Aroyo, Lora, Slone, Ambrose, Houlsby, Neil, Xiong, Xuehan, Yang, Zhen, Gribovskaya, Elena, Adler, Jonas, Wirth, Mateo, Lee, Lisa, Li, Music, Kagohara, Thais, Pavagadhi, Jay, Bridgers, Sophie, Bortsova, Anna, Ghemawat, Sanjay, Ahmed, Zafarali, Liu, Tianqi, Powell, Richard, Bolina, Vijay, Iinuma, Mariko, Zablotskaia, Polina, Besley, James, Chung, Da-Woon, Dozat, Timothy, Comanescu, Ramona, Si, Xiance, Greer, Jeremy, Su, Guolong, Polacek, Martin, Kaufman, Raphaël Lopez, Tokumine, Simon, Hu, Hexiang, Buchatskaya, Elena, Miao, Yingjie, Elhawaty, Mohamed, Siddhant, Aditya, Tomasev, Nenad, Xing, Jinwei, Greer, Christina, Miller, Helen, Ashraf, Shereen, Roy, Aurko, Zhang, Zizhao, Ma, Ada, Filos, Angelos, Besta, Milos, Blevins, Rory, Klimenko, Ted, Yeh, Chih-Kuan, Changpinyo, Soravit, Mu, Jiaqi, Chang, Oscar, Pajarskas, Mantas, Muir, Carrie, Cohen, Vered, Lan, Charline Le, Haridasan, Krishna, Marathe, Amit, Hansen, Steven, Douglas, Sholto, Samuel, Rajkumar, Wang, Mingqiu, Austin, Sophia, Lan, Chang, Jiang, Jiepu, Chiu, Justin, Lorenzo, Jaime Alonso, Sjösund, Lars Lowe, Cevey, Sébastien, Gleicher, Zach, Avrahami, Thi, Boral, Anudhyan, Srinivasan, Hansa, Selo, Vittorio, May, Rhys, Aisopos, Konstantinos, Hussenot, Léonard, Soares, Livio Baldini, Baumli, Kate, Chang, Michael B., Recasens, Adrià, Caine, Ben, Pritzel, Alexander, Pavetic, Filip, Pardo, Fabio, Gergely, Anita, Frye, Justin, Ramasesh, Vinay, Horgan, Dan, Badola, Kartikeya, Kassner, Nora, Roy, Subhrajit, Dyer, Ethan, Campos, Víctor Campos, Tomala, Alex, Tang, Yunhao, Badawy, Dalia El, White, Elspeth, Mustafa, Basil, Lang, Oran, Jindal, Abhishek, Vikram, Sharad, Gong, Zhitao, Caelles, Sergi, Hemsley, Ross, Thornton, Gregory, Feng, Fangxiaoyu, Stokowiec, Wojciech, Zheng, Ce, Thacker, Phoebe, Ünlü, Çağlar, Zhang, Zhishuai, Saleh, Mohammad, Svensson, James, Bileschi, Max, Patil, Piyush, Anand, Ankesh, Ring, Roman, Tsihlas, Katerina, Vezer, Arpi, Selvi, Marco, Shevlane, Toby, Rodriguez, Mikel, Kwiatkowski, Tom, Daruki, Samira, Rong, Keran, Dafoe, Allan, FitzGerald, Nicholas, Gu-Lemberg, Keren, Khan, Mina, Hendricks, Lisa Anne, Pellat, Marie, Feinberg, Vladimir, Cobon-Kerr, James, Sainath, Tara, Rauh, Maribeth, Hashemi, Sayed Hadi, Ives, Richard, Hasson, Yana, Noland, Eric, Cao, Yuan, Byrd, Nathan, Hou, Le, Wang, Qingze, Sottiaux, Thibault, Paganini, Michela, Lespiau, Jean-Baptiste, Moufarek, Alexandre, Hassan, Samer, Shivakumar, Kaushik, van Amersfoort, Joost, Mandhane, Amol, Joshi, Pratik, Goyal, Anirudh, Tung, Matthew, Brock, Andrew, Sheahan, Hannah, Misra, Vedant, Li, Cheng, Rakićević, Nemanja, Dehghani, Mostafa, Liu, Fangyu, Mittal, Sid, Oh, Junhyuk, Noury, Seb, Sezener, Eren, Huot, Fantine, Lamm, Matthew, De Cao, Nicola, Chen, Charlie, Mudgal, Sidharth, Stella, Romina, Brooks, Kevin, Vasudevan, Gautam, Liu, Chenxi, Chain, Mainak, Melinkeri, Nivedita, Cohen, Aaron, Wang, Venus, Seymore, Kristie, Zubkov, Sergey, Goel, Rahul, Yue, Summer, Krishnakumaran, Sai, Albert, Brian, Hurley, Nate, Sano, Motoki, Mohananey, Anhad, Joughin, Jonah, Filonov, Egor, Kępa, Tomasz, Eldawy, Yomna, Lim, Jiawern, Rishi, Rahul, Badiezadegan, Shirin, Bos, Taylor, Chang, Jerry, Jain, Sanil, Padmanabhan, Sri Gayatri Sundara, Puttagunta, Subha, Krishna, Kalpesh, Baker, Leslie, Kalb, Norbert, Bedapudi, Vamsi, Kurzrok, Adam, Lei, Shuntong, Yu, Anthony, Litvin, Oren, Zhou, Xiang, Wu, Zhichun, Sobell, Sam, Siciliano, Andrea, Papir, Alan, Neale, Robby, Bragagnolo, Jonas, Toor, Tej, Chen, Tina, Anklin, Valentin, Wang, Feiran, Feng, Richie, Gholami, Milad, Ling, Kevin, Liu, Lijuan, Walter, Jules, Moghaddam, Hamid, Kishore, Arun, Adamek, Jakub, Mercado, Tyler, Mallinson, Jonathan, Wandekar, Siddhinita, Cagle, Stephen, Ofek, Eran, Garrido, Guillermo, Lombriser, Clemens, Mukha, Maksim, Sun, Botu, Mohammad, Hafeezul Rahman, Matak, Josip, Qian, Yadi, Peswani, Vikas, Janus, Pawel, Yuan, Quan, Schelin, Leif, David, Oana, Garg, Ankur, He, Yifan, Duzhyi, Oleksii, Älgmyr, Anton, Lottaz, Timothée, Li, Qi, Yadav, Vikas, Xu, Luyao, Chinien, Alex, Shivanna, Rakesh, Chuklin, Aleksandr, Li, Josie, Spadine, Carrie, Wolfe, Travis, Mohamed, Kareem, Das, Subhabrata, Dai, Zihang, He, Kyle, von Dincklage, Daniel, Upadhyay, Shyam, Maurya, Akanksha, Chi, Luyan, Krause, Sebastian, Salama, Khalid, Rabinovitch, Pam G, M, Pavan Kumar Reddy, Selvan, Aarush, Dektiarev, Mikhail, Ghiasi, Golnaz, Guven, Erdem, Gupta, Himanshu, Liu, Boyi, Sharma, Deepak, Shtacher, Idan Heimlich, Paul, Shachi, Akerlund, Oscar, Aubet, François-Xavier, Huang, Terry, Zhu, Chen, Zhu, Eric, Teixeira, Elico, Fritze, Matthew, Bertolini, Francesco, Marinescu, Liana-Eleonora, Bölle, Martin, Paulus, Dominik, Gupta, Khyatti, Latkar, Tejasi, Chang, Max, Sanders, Jason, Wilson, Roopa, Wu, Xuewei, Tan, Yi-Xuan, Thiet, Lam Nguyen, Doshi, Tulsee, Lall, Sid, Mishra, Swaroop, Chen, Wanming, Luong, Thang, Benjamin, Seth, Lee, Jasmine, Andrejczuk, Ewa, Rabiej, Dominik, Ranjan, Vipul, Styrc, Krzysztof, Yin, Pengcheng, Simon, Jon, Harriott, Malcolm Rose, Bansal, Mudit, Robsky, Alexei, Bacon, Geoff, Greene, David, Mirylenka, Daniil, Zhou, Chen, Sarvana, Obaid, Goyal, Abhimanyu, Andermatt, Samuel, Siegler, Patrick, Horn, Ben, Israel, Assaf, Pongetti, Francesco, Chen, Chih-Wei "Louis", Selvatici, Marco, Silva, Pedro, Wang, Kathie, Tolins, Jackson, Guu, Kelvin, Yogev, Roey, Cai, Xiaochen, Agostini, Alessandro, Shah, Maulik, Nguyen, Hung, Donnaile, Noah Ó, Pereira, Sébastien, Friso, Linda, Stambler, Adam, Kuang, Chenkai, Romanikhin, Yan, Geller, Mark, Yan, ZJ, Jang, Kane, Lee, Cheng-Chun, Fica, Wojciech, Malmi, Eric, Tan, Qijun, Banica, Dan, Balle, Daniel, Pham, Ryan, Huang, Yanping, Avram, Diana, Shi, Hongzhi, Singh, Jasjot, Hidey, Chris, Ahuja, Niharika, Saxena, Pranab, Dooley, Dan, Potharaju, Srividya Pranavi, O'Neill, Eileen, Gokulchandran, Anand, Foley, Ryan, Zhao, Kai, Dusenberry, Mike, Liu, Yuan, Mehta, Pulkit, Kotikalapudi, Ragha, Safranek-Shrader, Chalence, Goodman, Andrew, Kessinger, Joshua, Globen, Eran, Kolhar, Prateek, Gorgolewski, Chris, Ibrahim, Ali, Song, Yang, Eichenbaum, Ali, Brovelli, Thomas, Potluri, Sahitya, Lahoti, Preethi, Baetu, Cip, Ghorbani, Ali, Chen, Charles, Crawford, Andy, Pal, Shalini, Sridhar, Mukund, Gurita, Petru, Mujika, Asier, Petrovski, Igor, Cedoz, Pierre-Louis, Li, Chenmei, Chen, Shiyuan, Santo, Niccolò Dal, Goyal, Siddharth, Punjabi, Jitesh, Kappaganthu, Karthik, Kwak, Chester, LV, Pallavi, Velury, Sarmishta, Choudhury, Himadri, Hall, Jamie, Shah, Premal, Figueira, Ricardo, Thomas, Matt, Lu, Minjie, Zhou, Ting, Kumar, Chintu, Jurdi, Thomas, Chikkerur, Sharat, Ma, Yenai, Yu, Adams, Kwak, Soo, Ähdel, Victor, Rajayogam, Sujeevan, Choma, Travis, Liu, Fei, Barua, Aditya, Ji, Colin, Park, Ji Ho, Hellendoorn, Vincent, Bailey, Alex, Bilal, Taylan, Zhou, Huanjie, Khatir, Mehrdad, Sutton, Charles, Rzadkowski, Wojciech, Macintosh, Fiona, Shagin, Konstantin, Medina, Paul, Liang, Chen, Zhou, Jinjing, Shah, Pararth, Bi, Yingying, Dankovics, Attila, Banga, Shipra, Lehmann, Sabine, Bredesen, Marissa, Lin, Zifan, Hoffmann, John Eric, Lai, Jonathan, Chung, Raynald, Yang, Kai, Balani, Nihal, Bražinskas, Arthur, Sozanschi, Andrei, Hayes, Matthew, Alcalde, Héctor Fernández, Makarov, Peter, Chen, Will, Stella, Antonio, Snijders, Liselotte, Mandl, Michael, Kärrman, Ante, Nowak, Paweł, Wu, Xinyi, Dyck, Alex, Vaidyanathan, Krishnan, R, Raghavender, Mallet, Jessica, Rudominer, Mitch, Johnston, Eric, Mittal, Sushil, Udathu, Akhil, Christensen, Janara, Verma, Vishal, Irving, Zach, Santucci, Andreas, Elsayed, Gamaleldin, Davoodi, Elnaz, Georgiev, Marin, Tenney, Ian, Hua, Nan, Cideron, Geoffrey, Leurent, Edouard, Alnahlawi, Mahmoud, Georgescu, Ionut, Wei, Nan, Zheng, Ivy, Scandinaro, Dylan, Jiang, Heinrich, Snoek, Jasper, Sundararajan, Mukund, Wang, Xuezhi, Ontiveros, Zack, Karo, Itay, Cole, Jeremy, Rajashekhar, Vinu, Tumeh, Lara, Ben-David, Eyal, Jain, Rishub, Uesato, Jonathan, Datta, Romina, Bunyan, Oskar, Wu, Shimu, Zhang, John, Stanczyk, Piotr, Zhang, Ye, Steiner, David, Naskar, Subhajit, Azzam, Michael, Johnson, Matthew, Paszke, Adam, Chiu, Chung-Cheng, Elias, Jaume Sanchez, Mohiuddin, Afroz, Muhammad, Faizan, Miao, Jin, Lee, Andrew, Vieillard, Nino, Park, Jane, Zhang, Jiageng, Stanway, Jeff, Garmon, Drew, Karmarkar, Abhijit, Dong, Zhe, Lee, Jong, Kumar, Aviral, Zhou, Luowei, Evens, Jonathan, Isaac, William, Irving, Geoffrey, Loper, Edward, Fink, Michael, Arkatkar, Isha, Chen, Nanxin, Shafran, Izhak, Petrychenko, Ivan, Chen, Zhe, Jia, Johnson, Levskaya, Anselm, Zhu, Zhenkai, Grabowski, Peter, Mao, Yu, Magni, Alberto, Yao, Kaisheng, Snaider, Javier, Casagrande, Norman, Palmer, Evan, Suganthan, Paul, Castaño, Alfonso, Giannoumis, Irene, Kim, Wooyeol, Rybiński, Mikołaj, Sreevatsa, Ashwin, Prendki, Jennifer, Soergel, David, Goedeckemeyer, Adrian, Gierke, Willi, Jafari, Mohsen, Gaba, Meenu, Wiesner, Jeremy, Wright, Diana Gage, Wei, Yawen, Vashisht, Harsha, Kulizhskaya, Yana, Hoover, Jay, Le, Maigo, Li, Lu, Iwuanyanwu, Chimezie, Liu, Lu, Ramirez, Kevin, Khorlin, Andrey, Cui, Albert, LIN, Tian, Wu, Marcus, Aguilar, Ricardo, Pallo, Keith, Chakladar, Abhishek, Perng, Ginger, Abellan, Elena Allica, Zhang, Mingyang, Dasgupta, Ishita, Kushman, Nate, Penchev, Ivo, Repina, Alena, Wu, Xihui, van der Weide, Tom, Ponnapalli, Priya, Kaplan, Caroline, Simsa, Jiri, Li, Shuangfeng, Dousse, Olivier, Piper, Jeff, Ie, Nathan, Pasumarthi, Rama, Lintz, Nathan, Vijayakumar, Anitha, Andor, Daniel, Valenzuela, Pedro, Lui, Minnie, Paduraru, Cosmin, Peng, Daiyi, Lee, Katherine, Zhang, Shuyuan, Greene, Somer, Nguyen, Duc Dung, Kurylowicz, Paula, Hardin, Cassidy, Dixon, Lucas, Janzer, Lili, Choo, Kiam, Feng, Ziqiang, Zhang, Biao, Singhal, Achintya, Du, Dayou, McKinnon, Dan, Antropova, Natasha, Bolukbasi, Tolga, Keller, Orgad, Reid, David, Finchelstein, Daniel, Raad, Maria Abi, Crocker, Remi, Hawkins, Peter, Dadashi, Robert, Gaffney, Colin, Franko, Ken, Bulanova, Anna, Leblond, Rémi, Chung, Shirley, Askham, Harry, Cobo, Luis C., Xu, Kelvin, Fischer, Felix, Xu, Jun, Sorokin, Christina, Alberti, Chris, Lin, Chu-Cheng, Evans, Colin, Dimitriev, Alek, Forbes, Hannah, Banarse, Dylan, Tung, Zora, Omernick, Mark, Bishop, Colton, Sterneck, Rachel, Jain, Rohan, Xia, Jiawei, Amid, Ehsan, Piccinno, Francesco, Wang, Xingyu, Banzal, Praseem, Mankowitz, Daniel J., Polozov, Alex, Krakovna, Victoria, Brown, Sasha, Bateni, MohammadHossein, Duan, Dennis, Firoiu, Vlad, Thotakuri, Meghana, Natan, Tom, Geist, Matthieu, Girgin, Ser tan, Li, Hui, Ye, Jiayu, Roval, Ofir, Tojo, Reiko, Kwong, Michael, Lee-Thorp, James, Yew, Christopher, Sinopalnikov, Danila, Ramos, Sabela, Mellor, John, Sharma, Abhishek, Wu, Kathy, Miller, David, Sonnerat, Nicolas, Vnukov, Denis, Greig, Rory, Beattie, Jennifer, Caveness, Emily, Bai, Libin, Eisenschlos, Julian, Korchemniy, Alex, Tsai, Tomy, Jasarevic, Mimi, Kong, Weize, Dao, Phuong, Zheng, Zeyu, Liu, Frederick, Zhu, Rui, Teh, Tian Huey, Sanmiya, Jason, Gladchenko, Evgeny, Trdin, Nejc, Toyama, Daniel, Rosen, Evan, Tavakkol, Sasan, Xue, Linting, Elkind, Chen, Woodman, Oliver, Carpenter, John, Papamakarios, George, Kemp, Rupert, Kafle, Sushant, Grunina, Tanya, Sinha, Rishika, Talbert, Alice, Wu, Diane, Owusu-Afriyie, Denese, Thornton, Chloe, Pont-Tuset, Jordi, Narayana, Pradyumna, Li, Jing, Fatehi, Saaber, Wieting, John, Ajmeri, Omar, Uria, Benigno, Ko, Yeongil, Knight, Laura, Héliou, Amélie, Niu, Ning, Gu, Shane, Pang, Chenxi, Li, Yeqing, Levine, Nir, Stolovich, Ariel, Santamaria-Fernandez, Rebeca, Goenka, Sonam, Yustalim, Wenny, Strudel, Robin, Elqursh, Ali, Deck, Charlie, Lee, Hyo, Li, Zonglin, Levin, Kyle, Hoffmann, Raphael, Holtmann-Rice, Dan, Bachem, Olivier, Arora, Sho, Koh, Christy, Yeganeh, Soheil Hassas, Põder, Siim, Tariq, Mukarram, Sun, Yanhua, Ionita, Lucian, Seyedhosseini, Mojtaba, Tafti, Pouya, Liu, Zhiyu, Gulati, Anmol, Liu, Jasmine, Ye, Xinyu, Chrzaszcz, Bart, Wang, Lily, Sethi, Nikhil, Li, Tianrun, Brown, Ben, Singh, Shreya, Fan, Wei, Parisi, Aaron, Stanton, Joe, Koverkathu, Vinod, Choquette-Choo, Christopher A., Li, Yunjie, Lu, TJ, Shroff, Prakash, Varadarajan, Mani, Bahargam, Sanaz, Willoughby, Rob, Gaddy, David, Desjardins, Guillaume, Cornero, Marco, Robenek, Brona, Mittal, Bhavishya, Albrecht, Ben, Shenoy, Ashish, Moiseev, Fedor, Jacobsson, Henrik, Ghaffarkhah, Alireza, Rivière, Morgane, Walton, Alanna, Crepy, Clément, Parrish, Alicia, Zhou, Zongwei, Farabet, Clement, Radebaugh, Carey, Srinivasan, Praveen, van der Salm, Claudia, Fidjeland, Andreas, Scellato, Salvatore, Latorre-Chimoto, Eri, Klimczak-Plucińska, Hanna, Bridson, David, de Cesare, Dario, Hudson, Tom, Mendolicchio, Piermaria, Walker, Lexi, Morris, Alex, Mauger, Matthew, Guseynov, Alexey, Reid, Alison, Odoom, Seth, Loher, Lucia, Cotruta, Victor, Yenugula, Madhavi, Grewe, Dominik, Petrushkina, Anastasia, Duerig, Tom, Sanchez, Antonio, Yadlowsky, Steve, Shen, Amy, Globerson, Amir, Webb, Lynette, Dua, Sahil, Li, Dong, Bhupatiraju, Surya, Hurt, Dan, Qureshi, Haroon, Agarwal, Ananth, Shani, Tomer, Eyal, Matan, Khare, Anuj, Belle, Shreyas Rammohan, Wang, Lei, Tekur, Chetan, Kale, Mihir Sanjay, Wei, Jinliang, Sang, Ruoxin, Saeta, Brennan, Liechty, Tyler, Sun, Yi, Zhao, Yao, Lee, Stephan, Nayak, Pandu, Fritz, Doug, Vuyyuru, Manish Reddy, Aslanides, John, Vyas, Nidhi, Wicke, Martin, Ma, Xiao, Eltyshev, Evgenii, Martin, Nina, Cate, Hardie, Manyika, James, Amiri, Keyvan, Kim, Yelin, Xiong, Xi, Kang, Kai, Luisier, Florian, Tripuraneni, Nilesh, Madras, David, Guo, Mandy, Waters, Austin, Wang, Oliver, Ainslie, Joshua, Baldridge, Jason, Zhang, Han, Pruthi, Garima, Bauer, Jakob, Yang, Feng, Mansour, Riham, Gelman, Jason, Xu, Yang, Polovets, George, Liu, Ji, Cai, Honglong, Chen, Warren, Sheng, XiangHai, Xue, Emily, Ozair, Sherjil, Angermueller, Christof, Li, Xiaowei, Sinha, Anoop, Wang, Weiren, Wiesinger, Julia, Koukoumidis, Emmanouil, Tian, Yuan, Iyer, Anand, Gurumurthy, Madhu, Goldenson, Mark, Shah, Parashar, Blake, MK, Yu, Hongkun, Urbanowicz, Anthony, Palomaki, Jennimaria, Fernando, Chrisantha, Durden, Ken, Mehta, Harsh, Momchev, Nikola, Rahimtoroghi, Elahe, Georgaki, Maria, Raul, Amit, Ruder, Sebastian, Redshaw, Morgan, Lee, Jinhyuk, Zhou, Denny, Jalan, Komal, Li, Dinghua, Hechtman, Blake, Schuh, Parker, Nasr, Milad, Milan, Kieran, Mikulik, Vladimir, Franco, Juliana, Green, Tim, Nguyen, Nam, Kelley, Joe, Mahendru, Aroma, Hu, Andrea, Howland, Joshua, Vargas, Ben, Hui, Jeffrey, Bansal, Kshitij, Rao, Vikram, Ghiya, Rakesh, Wang, Emma, Ye, Ke, Sarr, Jean Michel, Preston, Melanie Moranski, Elish, Madeleine, Li, Steve, Kaku, Aakash, Gupta, Jigar, Pasupat, Ice, Juan, Da-Cheng, Someswar, Milan, M., Tejvi, Chen, Xinyun, Amini, Aida, Fabrikant, Alex, Chu, Eric, Dong, Xuanyi, Muthal, Amruta, Buthpitiya, Senaka, Jauhari, Sarthak, Khandelwal, Urvashi, Hitron, Ayal, Ren, Jie, Rinaldi, Larissa, Drath, Shahar, Dabush, Avigail, Jiang, Nan-Jiang, Godhia, Harshal, Sachs, Uli, Chen, Anthony, Fan, Yicheng, Taitelbaum, Hagai, Noga, Hila, Dai, Zhuyun, Wang, James, Hamer, Jenny, Ferng, Chun-Sung, Elkind, Chenel, Atias, Aviel, Lee, Paulina, Listík, Vít, Carlen, Mathias, van de Kerkhof, Jan, Pikus, Marcin, Zaher, Krunoslav, Müller, Paul, Zykova, Sasha, Stefanec, Richard, Gatsko, Vitaly, Hirnschall, Christoph, Sethi, Ashwin, Xu, Xingyu Federico, Ahuja, Chetan, Tsai, Beth, Stefanoiu, Anca, Feng, Bo, Dhandhania, Keshav, Katyal, Manish, Gupta, Akshay, Parulekar, Atharva, Pitta, Divya, Zhao, Jing, Bhatia, Vivaan, Bhavnani, Yashodha, Alhadlaq, Omar, Li, Xiaolin, Danenberg, Peter, Tu, Dennis, Pine, Alex, Filippova, Vera, Ghosh, Abhipso, Limonchik, Ben, Urala, Bhargava, Lanka, Chaitanya Krishna, Clive, Derik, Li, Edward, Wu, Hao, Hongtongsak, Kevin, Li, Ianna, Thakkar, Kalind, Omarov, Kuanysh, Majmundar, Kushal, Alverson, Michael, Kucharski, Michael, Patel, Mohak, Jain, Mudit, Zabelin, Maksim, Pelagatti, Paolo, Kohli, Rohan, Kumar, Saurabh, Kim, Joseph, Sankar, Swetha, Shah, Vineet, Ramachandruni, Lakshmi, Zeng, Xiangkai, Bariach, Ben, Weidinger, Laura, Vu, Tu, Andreev, Alek, He, Antoine, Hui, Kevin, Kashem, Sheleem, Subramanya, Amar, Hsiao, Sissie, Hassabis, Demis, Kavukcuoglu, Koray, Sadovsky, Adam, Le, Quoc, Strohman, Trevor, Wu, Yonghui, Petrov, Slav, Dean, Jeffrey, and Vinyals, Oriol

Subjects
Computer Science - Computation and Language, Computer Science - Artificial Intelligence, Computer Science - Computer Vision and Pattern Recognition
Abstract

This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding. The Gemini family consists of Ultra, Pro, and Nano sizes, suitable for applications ranging from complex reasoning tasks to on-device memory-constrained use-cases. Evaluation on a broad range of benchmarks shows that our most-capable Gemini Ultra model advances the state of the art in 30 of 32 of these benchmarks - notably being the first model to achieve human-expert performance on the well-studied exam benchmark MMLU, and improving the state of the art in every one of the 20 multimodal benchmarks we examined. We believe that the new capabilities of the Gemini family in cross-modal reasoning and language understanding will enable a wide variety of use cases. We discuss our approach toward post-training and deploying Gemini models responsibly to users through services including Gemini, Gemini Advanced, Google AI Studio, and Cloud Vertex AI.

Published
2023

15. Tutorial: How to build and control an all-fiber wavefront modulator using mechanical perturbations

Author

Shekel, Ronen, Sulimany, Kfir, Resisi, Shachar, Finkelstein, Zohar, Lib, Ohad, Popoff, Sébastien M., and Bromberg, Yaron

Subjects
Physics - Optics, Physics - Instrumentation and Detectors
Abstract

Multimode optical fibers support the dense, low-loss transmission of many spatial modes, making them attractive for technologies such as communications and imaging. However, information propagating through multimode fibers is scrambled, due to modal dispersion and mode mixing. This is usually rectified using wavefront shaping techniques with devices such as spatial light modulators. Recently, we demonstrated an all-fiber system for controlling light propagation inside multimode fibers using mechanical perturbations, called the fiber piano. In this tutorial we explain the design considerations and experimental methods needed to build a fiber piano, and review applications where fiber pianos have been used.

Published
2023
Full Text
View/download PDF

16. Learning and Avoiding Disorder in Multimode Fibers

Author

Maxime W. Matthès, Yaron Bromberg, Julien de Rosny, and Sébastien M. Popoff

Subjects
Physics, QC1-999
Abstract

Multimode optical fibers (MMFs) have gained renewed interest in the past decade, emerging as a way to boost optical communication data rates in the context of an expected saturation of current single-mode fiber-based networks. They are also attractive for endoscopic applications, offering the possibility to achieve a similar information content as multicore fibers, but with a much smaller footprint, thus reducing the invasiveness of endoscopic procedures. However, these advances are hindered by the unavoidable presence of disorder that affects the propagation of light in MMFs and limits their practical applications. We introduce here a general framework to study and avoid the effect of disorder in wave-based systems and demonstrate its application for multimode fibers. We experimentally find an almost complete set of optical channels that are resilient to disorder induced by strong deformations. These deformation principal modes are obtained by only exploiting measurements for weak perturbations harnessing the generalized Wigner-Smith operator. We explain this effect by demonstrating that, even for a high level of disorder, the propagation of light in MMFs can be characterized by just a few key properties. These results are made possible thanks to a precise and fast estimation of the modal transmission matrix of the fiber which relies on a model-based optimization using deep learning frameworks.

Published
2021
Full Text
View/download PDF

17. Loss of hepatic DEPTOR alters the metabolic transition to fasting

Author

Alexandre Caron, Mathilde Mouchiroud, Nicolas Gautier, Sébastien M. Labbé, Romain Villot, Laurie Turcotte, Blandine Secco, Guillaume Lamoureux, Michael Shum, Yves Gélinas, André Marette, Denis Richard, David M. Sabatini, and Mathieu Laplante

Subjects
Internal medicine, RC31-1245
Abstract

Objective: The mechanistic target of rapamycin (mTOR) is a serine/threonine kinase that functions into distinct protein complexes (mTORC1 and mTORC2) that regulates growth and metabolism. DEP-domain containing mTOR-interacting protein (DEPTOR) is part of these complexes and is known to reduce their activity. Whether DEPTOR loss affects metabolism and organismal growth in vivo has never been tested. Methods: We have generated a conditional transgenic mouse allowing the tissue-specific deletion of DEPTOR. This model was crossed with CMV-cre mice or Albumin-cre mice to generate either whole-body or liver-specific DEPTOR knockout (KO) mice. Results: Whole-body DEPTOR KO mice are viable, fertile, normal in size, and do not display any gross physical and metabolic abnormalities. To circumvent possible compensatory mechanisms linked to the early and systemic loss of DEPTOR, we have deleted DEPTOR specifically in the liver, a tissue in which DEPTOR protein is expressed and affected in response to mTOR activation. Liver-specific DEPTOR null mice showed a reduction in circulating glucose upon fasting versus control mice. This effect was not associated with change in hepatic gluconeogenesis potential but was linked to a sustained reduction in circulating glucose during insulin tolerance tests. In addition to the reduction in glycemia, liver-specific DEPTOR KO mice had reduced hepatic glycogen content when fasted. We showed that loss of DEPTOR cell-autonomously increased oxidative metabolism in hepatocytes, an effect associated with increased cytochrome c expression but independent of changes in mitochondrial content or in the expression of genes controlling oxidative metabolism. We found that liver-specific DEPTOR KO mice showed sustained mTORC1 activation upon fasting, and that acute treatment with rapamycin was sufficient to normalize glycemia in these mice. Conclusion: We propose a model in which hepatic DEPTOR accelerates the inhibition of mTORC1 during the transition to fasting to adjust metabolism to the nutritional status. Keywords: DEPTOR, mTOR, Liver, Glucose, Fasting

Published
2017
Full Text
View/download PDF

18. A practical guide to Digital Micro-mirror Devices (DMDs) for wavefront shaping

Author

Popoff, Sébastien M., Bromberg, Yaron, Matthès, Maxime W., and Gutiérrez-Cuevas, Rodrigo

Subjects
Physics - Optics
Abstract

Digital micromirror devices have gained popularity in wavefront shaping, offering a high frame rate alternative to liquid crystal spatial light modulators. They are relatively inexpensive, offer high resolution, are easy to operate, and a single device can be used in a broad optical bandwidth. However, some technical drawbacks must be considered to achieve optimal performance. These issues, often undocumented by manufacturers, mostly stem from the device's original design for video projection applications. Herein, we present a guide to characterize and mitigate these effects. Our focus is on providing simple and practical solutions that can be easily incorporated into a typical wavefront shaping setup., Comment: 23 pages, 16 figures

Published
2023

19. Characterization and Exploitation of the Rotational Memory Effect in Multimode Fibers

Author

Gutiérrez-Cuevas, Rodrigo, Goetschy, Arthur, Bromberg, Yaron, Pelc, Guy, Andresen, Esben Ravn, Bigot, Laurent, Quiquempois, Yves, Bsaibes, Maroun, Sillard, Pierre, Bigot, Marianne, Katz, Ori, de Rosny, Julien, and Popoff, Sébastien M.

Subjects
Physics - Optics
Abstract

In an ideal perfectly straight multimode fiber with a circular-core, the symmetry ensures that rotating the input wavefront leads to a corresponding rotation of the output wavefront. This invariant property, known as the rotational memory effect (RME), remains independent of the typically unknown output profile. The RME thus offers significant potential for imaging and telecommunication applications. However, in real-life fibers, this effect is degraded by intrinsic imperfections and external perturbations, and is challenging to observe because of its acute sensitivity to misalignments and aberrations in the optical setup. Thanks to processing involving a spatial light modulator, we efficiently overcome these measurement biases, allowing for precise quantification of the RME. We establish an experimental and theoretical framework for studying and manipulating the RME in multimode fibers. Theoretical predictions are consistent with experimental data and simulations, connecting the shape of the angular-dependent correlation of the RME to the geometrical properties of the core deformation. This work opens the road for accurate characterization of the distributed disorder originating from the fabrication process and calibration-less imaging in multimode fibers.

Published
2023

20. Crashing with disorder: Reaching the precision limit with tensor-based wavefront shaping

Author

Gutiérrez-Cuevas, Rodrigo, Bouchet, Dorian, de Rosny, Julien, and Popoff, Sébastien M.

Subjects
Physics - Optics
Abstract

Perturbations in complex media, due to their own dynamical evolution or to external effects, are often seen as detrimental. Therefore, a common strategy, especially for telecommunication and imaging applications, is to limit the sensitivity to those perturbations in order to avoid them. Here, we instead consider crashing straight into them in order to maximize the interaction between light and the perturbations and thus produce the largest change in output intensity. Our work hinges on the innovative use of tensor-based techniques, presently at the forefront of machine learning explorations, to study intensity-based measurements where its quadratic relationship to the field prevents the use of standard matrix methods. With this tensor-based framework, we are able to identify the optimal crashing channel which maximizes the change in its output intensity distribution and the Fisher information encoded in it about a given perturbation. We further demonstrate experimentally its superiority for robust and precise sensing applications. Additionally, we derive the appropriate strategy to reach the precision limit for intensity-based measurements leading to an increase in Fisher information by more than four orders of magnitude with respect to the mean for random wavefronts when measured with the pixels of a camera.

Published
2023

21. RoboCLIP: One Demonstration is Enough to Learn Robot Policies

Author

Sontakke, Sumedh A, Zhang, Jesse, Arnold, Sébastien M. R., Pertsch, Karl, Bıyık, Erdem, Sadigh, Dorsa, Finn, Chelsea, and Itti, Laurent

Subjects
Computer Science - Artificial Intelligence, Computer Science - Robotics
Abstract

Reward specification is a notoriously difficult problem in reinforcement learning, requiring extensive expert supervision to design robust reward functions. Imitation learning (IL) methods attempt to circumvent these problems by utilizing expert demonstrations but typically require a large number of in-domain expert demonstrations. Inspired by advances in the field of Video-and-Language Models (VLMs), we present RoboCLIP, an online imitation learning method that uses a single demonstration (overcoming the large data requirement) in the form of a video demonstration or a textual description of the task to generate rewards without manual reward function design. Additionally, RoboCLIP can also utilize out-of-domain demonstrations, like videos of humans solving the task for reward generation, circumventing the need to have the same demonstration and deployment domains. RoboCLIP utilizes pretrained VLMs without any finetuning for reward generation. Reinforcement learning agents trained with RoboCLIP rewards demonstrate 2-3 times higher zero-shot performance than competing imitation learning methods on downstream robot manipulation tasks, doing so using only one video/text demonstration.

Published
2023

22. Mediobasal hypothalamic overexpression of DEPTOR protects against high-fat diet-induced obesity

Author

Alexandre Caron, Sébastien M. Labbé, Damien Lanfray, Pierre-Gilles Blanchard, Romain Villot, Christian Roy, David M. Sabatini, Denis Richard, and Mathieu Laplante

Subjects
Internal medicine, RC31-1245
Abstract

Background/Objective: The mechanistic target of rapamycin (mTOR) is a serine–threonine kinase that functions into distinct protein complexes (mTORC1 and mTORC2) that regulate energy homeostasis. DEP-domain containing mTOR-interacting protein (DEPTOR) is part of these complexes and is known to dampen mTORC1 function, consequently reducing mTORC1 negative feedbacks and promoting insulin signaling and Akt/PKB activation in several models. Recently, we observed that DEPTOR is expressed in several structures of the brain including the mediobasal hypothalamus (MBH), a region that regulates energy balance. Whether DEPTOR in the MBH plays a functional role in regulating energy balance and hypothalamic insulin signaling has never been tested. Methods: We have generated a novel conditional transgenic mouse model based on the Cre-LoxP system allowing targeted overexpression of DEPTOR. Mice overexpressing DEPTOR in the MBH were subjected to a metabolic phenotyping and MBH insulin signaling was evaluated. Results: We first report that systemic (brain and periphery) overexpression of DEPTOR prevents high-fat diet-induced obesity, improves glucose metabolism and protects against hepatic steatosis. These phenotypes were associated with a reduction in food intake and feed efficiency and an elevation in oxygen consumption. Strikingly, specific overexpression of DEPTOR in the MBH completely recapitulated these phenotypes. DEPTOR overexpression was associated with an increase in hypothalamic insulin signaling, as illustrated by elevated Akt/PKB activation. Conclusion: Altogether, these results support a role for MBH DEPTOR in the regulation of energy balance and metabolism. Keywords: mTOR, DEPTOR, Hypothalamus, Energy balance, Glucose metabolism

Published
2016
Full Text
View/download PDF

23. Dynamic structured illumination for confocal microscopy

Author

Noetinger, Guillaume, Lemoult, Fabrice, and Popoff, Sébastien M.

Subjects
Physics - Optics, Physics - Applied Physics
Abstract

Structured illumination enables the tailoring of an imaging device's optical transfer function to enhance resolution. We propose the incorporation of a temporal periodic modulation, specifically a rotating mask, to encode multiple transfer functions in the temporal domain. This approach is demonstrated using a confocal microscope configuration. At each scanning position, a temporal periodic signal is recorded. By filtering around each harmonic of the rotation frequency, multiple images of the same object can be constructed. The image carried by the $n{\mathrm{th}}$ harmonic is a convolution of the object with a phase vortex of topological charge $n$, similar to the outcome when using a vortex phase plate as an illumination. This enables the collection of chosen high spatial frequencies from the sample, thereby enhancing the spatial resolution of the confocal microscope., Comment: 4 pages of main text with 4 figures, 14 pages of SI

Published
2023

24. Shaping Single Photons through Multimode Optical Fibers using Mechanical Perturbations

Author

Shekel, Ronen, Lib, Ohad, Gutiérrez-Cuevas, Rodrigo, Popoff, Sébastien M., Ling, Alexander, and Bromberg, Yaron

Subjects
Quantum Physics, Physics - Optics
Abstract

The capacity of information delivered by single photons is boosted by encoding high-dimensional quantum dits in their transverse shape. Transporting such high-dimensional quantum dits in optical networks may be accomplished using multimode optical fibers, which support the low-loss transmission of multiple spatial modes over existing infrastructure. However, when photons propagate through a multimode fiber their transverse shape gets scrambled because of mode mixing and modal interference. This is usually corrected using free-space spatial light modulators, inhibiting a robust all-fiber operation. In this work, we demonstrate an all-fiber approach for controlling the shape of single photons and the spatial correlations between entangled photon pairs, using carefully controlled mechanical perturbations of the fiber. We optimize these perturbations to localize the spatial distribution of a single photon or the spatial correlations of photon pairs in a single spot, enhancing the signal in the optimized spot by over an order of magnitude. Using the same approach we show a similar enhancement for coupling light from a multimode fiber into a single-mode fiber.

Published
2023
Full Text
View/download PDF

26. Policy-Induced Self-Supervision Improves Representation Finetuning in Visual RL

Author

Arnold, Sébastien M. R. and Sha, Fei

Subjects
Computer Science - Machine Learning, Computer Science - Computer Vision and Pattern Recognition
Abstract

We study how to transfer representations pretrained on source tasks to target tasks in visual percept based RL. We analyze two popular approaches: freezing or finetuning the pretrained representations. Empirical studies on a set of popular tasks reveal several properties of pretrained representations. First, finetuning is required even when pretrained representations perfectly capture the information required to solve the target task. Second, finetuned representations improve learnability and are more robust to noise. Third, pretrained bottom layers are task-agnostic and readily transferable to new tasks, while top layers encode task-specific information and require adaptation. Building on these insights, we propose a self-supervised objective that clusters representations according to the policy they induce, as opposed to traditional representation similarity measures which are policy-agnostic (e.g. Euclidean norm, cosine similarity). Together with freezing the bottom layers, this objective results in significantly better representation than frozen, finetuned, and self-supervised alternatives on a wide range of benchmarks.

Published
2023

27. A Domain-Agnostic Approach for Characterization of Lifelong Learning Systems

Author

Baker, Megan M., New, Alexander, Aguilar-Simon, Mario, Al-Halah, Ziad, Arnold, Sébastien M. R., Ben-Iwhiwhu, Ese, Brna, Andrew P., Brooks, Ethan, Brown, Ryan C., Daniels, Zachary, Daram, Anurag, Delattre, Fabien, Dellana, Ryan, Eaton, Eric, Fu, Haotian, Grauman, Kristen, Hostetler, Jesse, Iqbal, Shariq, Kent, Cassandra, Ketz, Nicholas, Kolouri, Soheil, Konidaris, George, Kudithipudi, Dhireesha, Learned-Miller, Erik, Lee, Seungwon, Littman, Michael L., Madireddy, Sandeep, Mendez, Jorge A., Nguyen, Eric Q., Piatko, Christine D., Pilly, Praveen K., Raghavan, Aswin, Rahman, Abrar, Ramakrishnan, Santhosh Kumar, Ratzlaff, Neale, Soltoggio, Andrea, Stone, Peter, Sur, Indranil, Tang, Zhipeng, Tiwari, Saket, Vedder, Kyle, Wang, Felix, Xu, Zifan, Yanguas-Gil, Angel, Yedidsion, Harel, Yu, Shangqun, and Vallabha, Gautam K.

Subjects
Computer Science - Machine Learning, Computer Science - Computer Vision and Pattern Recognition, Computer Science - Robotics
Abstract

Despite the advancement of machine learning techniques in recent years, state-of-the-art systems lack robustness to "real world" events, where the input distributions and tasks encountered by the deployed systems will not be limited to the original training context, and systems will instead need to adapt to novel distributions and tasks while deployed. This critical gap may be addressed through the development of "Lifelong Learning" systems that are capable of 1) Continuous Learning, 2) Transfer and Adaptation, and 3) Scalability. Unfortunately, efforts to improve these capabilities are typically treated as distinct areas of research that are assessed independently, without regard to the impact of each separate capability on other aspects of the system. We instead propose a holistic approach, using a suite of metrics and an evaluation framework to assess Lifelong Learning in a principled way that is agnostic to specific domains or system techniques. Through five case studies, we show that this suite of metrics can inform the development of varied and complex Lifelong Learning systems. We highlight how the proposed suite of metrics quantifies performance trade-offs present during Lifelong Learning system development - both the widely discussed Stability-Plasticity dilemma and the newly proposed relationship between Sample Efficient and Robust Learning. Further, we make recommendations for the formulation and use of metrics to guide the continuing development of Lifelong Learning systems and assess their progress in the future., Comment: To appear in Neural Networks

Published
2023
Full Text
View/download PDF

28. TP53 drives invasion through expression of its Δ133p53β variant

Author

Gilles Gadea, Nikola Arsic, Kenneth Fernandes, Alexandra Diot, Sébastien M Joruiz, Samer Abdallah, Valerie Meuray, Stéphanie Vinot, Christelle Anguille, Judit Remenyi, Marie P Khoury, Philip R Quinlan, Colin A Purdie, Lee B Jordan, Frances V Fuller-Pace, Marion de Toledo, Maïlys Cren, Alastair M Thompson, Jean-Christophe Bourdon, and Pierre Roux

Subjects
p53 isoforms, cancer invasion, EMT, survival, Medicine, Science, Biology (General), QH301-705.5
Abstract

TP53 is conventionally thought to prevent cancer formation and progression to metastasis, while mutant TP53 has transforming activities. However, in the clinic, TP53 mutation status does not accurately predict cancer progression. Here we report, based on clinical analysis corroborated with experimental data, that the p53 isoform Δ133p53β promotes cancer cell invasion, regardless of TP53 mutation status. Δ133p53β increases risk of cancer recurrence and death in breast cancer patients. Furthermore Δ133p53β is critical to define invasiveness in a panel of breast and colon cell lines, expressing WT or mutant TP53. Endogenous mutant Δ133p53β depletion prevents invasiveness without affecting mutant full-length p53 protein expression. Mechanistically WT and mutant Δ133p53β induces EMT. Our findings provide explanations to 2 long-lasting and important clinical conundrums: how WT TP53 can promote cancer cell invasion and reciprocally why mutant TP53 gene does not systematically induce cancer progression.

Published
2016
Full Text
View/download PDF

29. Event-Based Eye Tracking. AIS 2024 Challenge Survey.

Author

Zuowen Wang, Chang Gao 0002, Zongwei Wu, Marcos V. Conde, Radu Timofte, Shih-Chii Liu, Qinyu Chen, Zhengjun Zha, Wei Zhai, Han Han, Bohao Liao, Yuliang Wu, Zengyu Wan, Zhong Wang, Yang Cao 0010, Ganchao Tan, Jinze Chen, Yan Ru Pei, Sasskia Brüers, Sébastien M. Crouzet, Douglas McLelland, Olivier Coenen, Baoheng Zhang, Yizhao Gao, Jingyuan Li, Hayden Kwok-Hay So, Philippe Bich, Chiara Boretti, Luciano Prono, Mircea Lica, David Dinucu-Jianu, Catalin Grîu, Xiaopeng Lin, Hongwei Ren, Bojun Cheng, Xinan Zhang, Valentin Vial, Anthony Yezzi, and James Tsai

Published
2024
Full Text
View/download PDF

32. Super-resolved imaging based on spatiotemporal wavefront shaping

Author

Noetinger, Guillaume, Métais, Samuel, Lerosey, Geoffroy, Fink, Mathias, Popoff, Sébastien M., and Lemoult, Fabrice

Subjects
Physics - Optics, Physics - Applied Physics, Physics - Classical Physics
Abstract

A novel approach to improving the performances of confocal scanning imaging is proposed. We experimentally demonstrate its feasibility using acoustic waves. It relies on a new way to encode spatial information using the temporal dimension. By moving an emitter, used to insonify an object, along a circular path, we create a temporally modulated wavefield. Due to the cylindrical symmetry of the problem and its temporal periodicity, the spatiotemporal input field can be decomposed into harmonics corresponding to different spatial vortices, or topological charges. Acquiring the back-reflected waves with receivers which are also rotating, multiple images of the same object with different Point Spread Functions (PSFs) are obtained. Not only is the resolution improved compared to a standard confocal configuration, but the accumulation of information also allows building images beating the diffraction limit. The topological robustness of the approach promises good performances in real life conditions., Comment: 5 pages, 4 figures Supplementary file

Published
2022

33. Explaining the Timing of Natural Scene Understanding with a Computational Model of Perceptual Categorization.

Author

Imri Sofer, Sébastien M Crouzet, and Thomas Serre

Subjects
Biology (General), QH301-705.5
Abstract

Observers can rapidly perform a variety of visual tasks such as categorizing a scene as open, as outdoor, or as a beach. Although we know that different tasks are typically associated with systematic differences in behavioral responses, to date, little is known about the underlying mechanisms. Here, we implemented a single integrated paradigm that links perceptual processes with categorization processes. Using a large image database of natural scenes, we trained machine-learning classifiers to derive quantitative measures of task-specific perceptual discriminability based on the distance between individual images and different categorization boundaries. We showed that the resulting discriminability measure accurately predicts variations in behavioral responses across categorization tasks and stimulus sets. We further used the model to design an experiment, which challenged previous interpretations of the so-called "superordinate advantage." Overall, our study suggests that observed differences in behavioral responses across rapid categorization tasks reflect natural variations in perceptual discriminability.

Published
2015
Full Text
View/download PDF

34. Restoring and tailoring very high dimensional spatial entanglement of a biphoton state transmitted through a scattering medium

Author

Devaux, Fabrice, Mosset, Alexis, Popoff, Sébastien M., and Lantz, Eric

Subjects
Quantum Physics, Physics - Optics
Abstract

We report experimental results where a momentum entangled biphoton state with a giant dimensionality of 8000 is retrieved and manipulated when only one photon of the pair is transmitted through a thin scattering medium. For this purpose, the transmission matrix of the complex medium is first measured with a phase-shifting interferometry measurement method using a spatial light modulator (SLM) illuminated with a laser source. From this matrix, different phase masks are calculated and addressed on the SLM to spatially control the focusing of the laser through the complex medium. These same masks are used to manipulate the phase of the biphoton wave function transmitted by the thin diffuser in order to restore and control in the same way the momentum correlations between the far-field images of twin beams issued from strongly spatial-multi-mode spontaneous parametric down conversion., Comment: 5 pages, 4 figures

Published
2022
Full Text
View/download PDF

35. The fastest saccadic responses escape visual masking.

Author

Sébastien M Crouzet, Morten Overgaard, and Niko A Busch

Subjects
Medicine, Science
Abstract

Object-substitution masking (OSM) occurs when a briefly presented target in a search array is surrounded by small dots that remain visible after the target disappears. The reduction of target visibility occurring after OSM has been suggested to result from a specific interference with reentrant visual processing while the initial feedforward processing is thought to be left intact. We tested a prediction derived from this hypothesis: the fastest responses, being triggered before the beginning of reentrant processing, should escape the OSM interference. In a saccadic choice reaction time task, which gives access to very early stages of visual processing, target visibility was reduced either by OSM, conventional backward masking, or low stimulus contrast. A general reduction of performance was observed in all three conditions. However, the fastest saccades did not show any sign of interference under either OSM or backward masking, as they did under the low-contrast condition. This finding supports the hypothesis that masking interferes mostly with reentrant processing at later stages, while leaving early feedforward processing largely intact.

Published
2014
Full Text
View/download PDF

36. Policy Learning and Evaluation with Randomized Quasi-Monte Carlo

Author

Arnold, Sebastien M. R., L'Ecuyer, Pierre, Chen, Liyu, Chen, Yi-fan, and Sha, Fei

Subjects
Computer Science - Machine Learning
Abstract

Reinforcement learning constantly deals with hard integrals, for example when computing expectations in policy evaluation and policy iteration. These integrals are rarely analytically solvable and typically estimated with the Monte Carlo method, which induces high variance in policy values and gradients. In this work, we propose to replace Monte Carlo samples with low-discrepancy point sets. We combine policy gradient methods with Randomized Quasi-Monte Carlo, yielding variance-reduced formulations of policy gradient and actor-critic algorithms. These formulations are effective for policy evaluation and policy improvement, as they outperform state-of-the-art algorithms on standardized continuous control benchmarks. Our empirical analyses validate the intuition that replacing Monte Carlo with Quasi-Monte Carlo yields significantly more accurate gradient estimates., Comment: AISTATS 2022 camera ready; more info at: http://seba1511.net/projects/qrl/

Published
2022

39. Control and physiological determinants of sympathetically-mediated brown adipose tissue thermogenesis

Author

Denis eRichard, Boris eMonge Roffarello, Kanta eChechi, Sébastien M Labbé, and Éric eTurcotte

Subjects
energy balance, age, Sex, cold exposure, Energy Expenditure, positron emission tomography, Diseases of the endocrine glands. Clinical endocrinology, RC648-665
Abstract

Brown adipose tissue (BAT) represents a remarkable heat-producing tissue. The thermogenic potential of BAT is conferred by uncoupling protein 1, a protein found uniquely in the brown adipocytes. The physiological control of BAT activity and capacity is ensured by the sympathetic nervous system (SNS), which densely innervates brown fat depots. SNS-mediated BAT thermogenesis is essentially governed by hypothalamic and brainstem neurons. BAT is not only controlled by the brain thermoregulatory circuits but also by brain energy balance pathways including the very significant brain melanocortin system, which speaks in favor of the genuine involvement of SNS-mediated BAT thermogenesis in energy homeostasis. The use of positron emission tomography/computed tomography (PET/CT) scanning has further revealed the presence of well-defined BAT depots in the cervical, clavicular, and paraspinal areas in adult humans. The prevalence of these depots was reported to be higher in subjects exposed to low temperature and was also higher in women than men. Moreover, the prevalence of BAT was shown to decrease with age and body fat mass, which suggests that BAT could not only be involved in cold-induced non shivering thermogenesis but also in the energy balance regulation and obesity in humans. This short review summarizes recent progress made in our understanding of the control of SNS-mediated BAT thermogenesis and of the determinants of BAT prevalence or detection in humans.

Published
2012
Full Text
View/download PDF

40. Animal detection precedes access to scene category.

Author

Sébastien M Crouzet, Olivier R Joubert, Simon J Thorpe, and Michèle Fabre-Thorpe

Subjects
Medicine, Science
Abstract

The processes underlying object recognition are fundamental for the understanding of visual perception. Humans can recognize many objects rapidly even in complex scenes, a task that still presents major challenges for computer vision systems. A common experimental demonstration of this ability is the rapid animal detection protocol, where human participants earliest responses to report the presence/absence of animals in natural scenes are observed at 250-270 ms latencies. One of the hypotheses to account for such speed is that people would not actually recognize an animal per se, but rather base their decision on global scene statistics. These global statistics (also referred to as spatial envelope or gist) have been shown to be computationally easy to process and could thus be used as a proxy for coarse object recognition. Here, using a saccadic choice task, which allows us to investigate a previously inaccessible temporal window of visual processing, we showed that animal - but not vehicle - detection clearly precedes scene categorization. This asynchrony is in addition validated by a late contextual modulation of animal detection, starting simultaneously with the availability of scene category. Interestingly, the advantage for animal over scene categorization is in opposition to the results of simulations using standard computational models. Taken together, these results challenge the idea that rapid animal detection might be based on early access of global scene statistics, and rather suggests a process based on the extraction of specific local complex features that might be hardwired in the visual system.

Published
2012
Full Text
View/download PDF

41. What are the visual features underlying rapid object recognition?

Author

Sébastien M Crouzet and Thomas eSerre

Subjects
Computational models, Computer Vision, feedforward, rapid visual object recognition, visual features, Psychology, BF1-990
Abstract

Research progress in machine vision has been very significant in recent years. Robust face detection and identification algorithms are already readily available to consumers, and modern computer vision algorithms for generic object recognition are now coping with the richness and complexity of natural visual scenes. Unlike early vision models of object recognition that emphasized the role of figure-ground segmentation and spatial information between parts, recent successful approaches are based on the computation of loose collections of image features without prior segmentation or any explicit encoding of spatial relations. While these models remain simplistic models of visual processing, they suggest that, in principle, bottom-up activation of a loose collection of image features could support the rapid recognition of natural object categories and provide an initial coarse visual representation before more complex visual routines and attentional mechanisms take place. Focusing on biologically-plausible computational models of (bottom-up) pre-attentive visual recognition, we review some of the key visual features that have been described in the literature. We discuss the consistency of these feature-based representations with classical theories from visual psychology and test their ability to account for human performance on a rapid object categorization task.

Published
2011
Full Text
View/download PDF

42. Low level cues and ultra-fast face detection

Author

Sébastien M Crouzet and Simon J Thorpe

Subjects
natural scenes, Amplitude Spectrum, Face detection, Fast Saccades, Fourier Transform, Psychology, BF1-990
Abstract

Recent experimental work has demonstrated the existence of extremely rapid saccades towards faces in natural scenes that can be initiated only 100 ms after image onset (Crouzet, Kirchner, & Thorpe, 2010). These ultra-rapid saccades constitute a major challenge to current models of processing in the visual system because they do not seem to leave enough time for even a single feed-forward pass through the ventral stream. Here we explore the possibility that the information required to trigger these very fast saccades could be extracted very early on in visual processing using relatively low-level amplitude spectrum (AS) information in the Fourier domain. Experiment 1 showed that AS normalization can significantly alter face detection performance. However, a decrease of performance following AS normalization does not alone prove that AS-based information is used (Gaspar & Rousselet, 2009). In Experiment 2, following the Gaspar and Rousselet paper, we used a swapping procedure to clarify the role of AS information in fast object detection. Our experiment is composed of 3 conditions: (i) original images, (ii) inverted, in which the face image has the AS of a vehicle, and the vehicle has the AS of a face, and (iii) swapped, where the face has the AS of another face image, and the vehicle has the AS of another vehicle image. The results showed very similar levels of performance in the original and swapped conditions, and a clear drop in the inverted condition. This result demonstrates that, in the early temporal window offered by the saccadic choice task, the visual saccadic system does indeed rely on low-level AS information in order to rapidly detect faces. This sort of crude diagnostic information could potentially be derived very early on in the visual system, possibly as early as V1 and V2.

Published
2011
Full Text
View/download PDF

43. Increased postprandial nonesterified fatty acid appearance and oxidation in type 2 diabetes is not fully established in offspring of diabetic subjects.

Author

François Normand-Lauzière, Frédérique Frisch, Sébastien M Labbé, Patrick Bherer, René Gagnon, Stephen C Cunnane, and André C Carpentier

Subjects
Medicine, Science
Abstract

It has been proposed that abnormal postprandial plasma nonesterified fatty acid (NEFA) metabolism may participate in the development of tissue lipotoxicity and type 2 diabetes (T2D). We previously found that non-diabetic offspring of two parents with T2D display increased plasma NEFA appearance and oxidation rates during intravenous administration of a fat emulsion. However, it is currently unknown whether plasma NEFA appearance and oxidation are abnormal during the postprandial state in these subjects at high-risk of developing T2D.Palmitate appearance and oxidation rates and glycerol appearance rate were determined in eleven healthy offspring of two parents with T2D (positive family history, FH+), 13 healthy subjects without first-degree relatives with T2D (FH-) and 12 subjects with T2D at fasting, during normoglycemic hyperinsulinemic clamp and during continuous oral intake of a standard liquid meal to achieve steady postprandial NEFA and triacylglycerols (TG) without and with insulin infusion to maintain similar glycemia in all three groups.Plasma palmitate appearance and oxidation were higher at fasting and during the clamp conditions in the T2D group (all P

Published
2010
Full Text
View/download PDF

45. Uniform Sampling over Episode Difficulty

Author

Arnold, Sébastien M. R., Dhillon, Guneet S., Ravichandran, Avinash, and Soatto, Stefano

Subjects
Computer Science - Machine Learning, Computer Science - Artificial Intelligence, Computer Science - Computer Vision and Pattern Recognition
Abstract

Episodic training is a core ingredient of few-shot learning to train models on tasks with limited labelled data. Despite its success, episodic training remains largely understudied, prompting us to ask the question: what is the best way to sample episodes? In this paper, we first propose a method to approximate episode sampling distributions based on their difficulty. Building on this method, we perform an extensive analysis and find that sampling uniformly over episode difficulty outperforms other sampling schemes, including curriculum and easy-/hard-mining. As the proposed sampling method is algorithm agnostic, we can leverage these insights to improve few-shot learning accuracies across many episodic training algorithms. We demonstrate the efficacy of our method across popular few-shot learning datasets, algorithms, network architectures, and protocols., Comment: NeurIPS'21 camera ready

Published
2021

46. Embedding Adaptation is Still Needed for Few-Shot Learning

Author

Arnold, Sébastien M. R. and Sha, Fei

Subjects
Computer Science - Machine Learning, Computer Science - Computer Vision and Pattern Recognition
Abstract

Constructing new and more challenging tasksets is a fruitful methodology to analyse and understand few-shot classification methods. Unfortunately, existing approaches to building those tasksets are somewhat unsatisfactory: they either assume train and test task distributions to be identical -- which leads to overly optimistic evaluations -- or take a "worst-case" philosophy -- which typically requires additional human labor such as obtaining semantic class relationships. We propose ATG, a principled clustering method to defining train and test tasksets without additional human knowledge. ATG models train and test task distributions while requiring them to share a predefined amount of information. We empirically demonstrate the effectiveness of ATG in generating tasksets that are easier, in-between, or harder than existing benchmarks, including those that rely on semantic information. Finally, we leverage our generated tasksets to shed a new light on few-shot classification: gradient-based methods -- previously believed to underperform -- can outperform metric-based ones when transfer is most challenging., Comment: In submission

Published
2021

48. Image transmission through a flexible multimode fiber by deep learning

Author

Resisi, Shachar, Popoff, Sebastien M., and Bromberg, Yaron

Subjects
Electrical Engineering and Systems Science - Image and Video Processing, Physics - Medical Physics, Physics - Optics
Abstract

When multimode optical fibers are perturbed, the data that is transmitted through them is scrambled. This presents a major difficulty for many possible applications, such as multimode fiber-based telecommunication and endoscopy. To overcome this challenge, a deep learning approach that generalizes over mechanical perturbations is presented. Using this approach, successful reconstruction of the input images from intensity-only measurements of speckle patterns at the output of a 1.5 meter-long randomly perturbed multimode fiber is demonstrated. The model's success is explained by hidden correlations in the speckle of random fiber conformations.

Published
2020
Full Text
View/download PDF

49. Learning and avoiding disorder in multimode fibers

Author

Matthès, Maxime W., Bromberg, Yaron, de Rosny, Julien, and Popoff, Sébastien M.

Subjects
Physics - Optics
Abstract

Multimode optical fibers (MMFs) have gained renewed interest in the past decade, emerging as a way to boost optical communication data-rates in the context of an expected saturation of current single-mode fiber-based networks. They are also attractive for endoscopic applications, offering the possibility to achieve a similar information content as multicore fibers, but with a much smaller footprint, thus reducing the invasiveness of endoscopic procedures. However, these advances are hindered by the unavoidable presence of disorder that affects the propagation of light in MMFs and limits their practical applications. We introduce here a general framework to study and avoid the effect of disorder. We experimentally find an almost complete set of optical channels that are resilient to disorder induced by strong deformations. These deformation principle modes are obtained by only exploiting measurements for weak perturbations. We explain this effect by demonstrating that, even for a high level of disorder, the propagation of light in MMFs can be characterized by just a few key properties. These results are made possible thanks to a precise and fast estimation of the modal transmission matrix of the fiber which relies on a model-based optimization using deep learning frameworks.

Published
2020
Full Text
View/download PDF

50. learn2learn: A Library for Meta-Learning Research

Author

Arnold, Sébastien M. R., Mahajan, Praateek, Datta, Debajyoti, Bunner, Ian, and Zarkias, Konstantinos Saitas

Subjects
Computer Science - Machine Learning, Computer Science - Computer Vision and Pattern Recognition, Computer Science - Robotics, Statistics - Machine Learning
Abstract

Meta-learning researchers face two fundamental issues in their empirical work: prototyping and reproducibility. Researchers are prone to make mistakes when prototyping new algorithms and tasks because modern meta-learning methods rely on unconventional functionalities of machine learning frameworks. In turn, reproducing existing results becomes a tedious endeavour -- a situation exacerbated by the lack of standardized implementations and benchmarks. As a result, researchers spend inordinate amounts of time on implementing software rather than understanding and developing new ideas. This manuscript introduces learn2learn, a library for meta-learning research focused on solving those prototyping and reproducibility issues. learn2learn provides low-level routines common across a wide-range of meta-learning techniques (e.g. meta-descent, meta-reinforcement learning, few-shot learning), and builds standardized interfaces to algorithms and benchmarks on top of them. In releasing learn2learn under a free and open source license, we hope to foster a community around standardized software for meta-learning research., Comment: Software available at: https://github.com/learnables/learn2learn

Published
2020

Catalog

Books, media, physical & digital resources
See catalog results