Search

In this report, we introduce the Gemini 1.5 family of models, representing the next generation of highly compute-efficient multimodal models capable of recalling and reasoning over fine-grained information from millions of tokens of context, including multiple long documents and hours of video and audio. The family includes two new models: (1) an updated Gemini 1.5 Pro, which exceeds the February version on the great majority of capabilities and benchmarks; (2) Gemini 1.5 Flash, a more lightweight variant designed for efficiency with minimal regression in quality. Gemini 1.5 models achieve near-perfect recall on long-context retrieval tasks across modalities, improve the state-of-the-art in long-document QA, long-video QA and long-context ASR, and match or surpass Gemini 1.0 Ultra's state-of-the-art performance across a broad set of benchmarks. Studying the limits of Gemini 1.5's long-context ability, we find continued improvement in next-token prediction and near-perfect retrieval (>99%) up to at least 10M tokens, a generational leap over existing models such as Claude 3.0 (200k) and GPT-4 Turbo (128k). Finally, we highlight real-world use cases, such as Gemini 1.5 collaborating with professionals on completing their tasks achieving 26 to 75% time savings across 10 different job categories, as well as surprising new capabilities of large language models at the frontier; when given a grammar manual for Kalamang, a language with fewer than 200 speakers worldwide, the model learns to translate English to Kalamang at a similar level to a person who learned from the same content.

This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding. The Gemini family consists of Ultra, Pro, and Nano sizes, suitable for applications ranging from complex reasoning tasks to on-device memory-constrained use-cases. Evaluation on a broad range of benchmarks shows that our most-capable Gemini Ultra model advances the state of the art in 30 of 32 of these benchmarks - notably being the first model to achieve human-expert performance on the well-studied exam benchmark MMLU, and improving the state of the art in every one of the 20 multimodal benchmarks we examined. We believe that the new capabilities of the Gemini family in cross-modal reasoning and language understanding will enable a wide variety of use cases. We discuss our approach toward post-training and deploying Gemini models responsibly to users through services including Gemini, Gemini Advanced, Google AI Studio, and Cloud Vertex AI.

We explore the state of matter characterized by the charged timelike vector order parameter. We employ holographic duality in order to construct such a state and study its thermoelectric transport, fermionic spectral function and the sign of the Meissner effect. We uncover the unusual features of this "timelike superconductor": the absence of the gap in the fermionic spectrum and co-existence of Drude peak and supercurrent in the AC transport, which are reminiscent to those of time-reversal-odd and gapless superconductors, correspondingly. We show that this state is dynamically stable and thermodynamically at least metastable. Along the way we develop the holographic model of the charged vector field, which acquires mass due to a variant of the Stueckelberg mechanism with the adjoint Higgs field., Comment: 18 pages plus appendices

We introduce PaLM 2, a new state-of-the-art language model that has better multilingual and reasoning capabilities and is more compute-efficient than its predecessor PaLM. PaLM 2 is a Transformer-based model trained using a mixture of objectives. Through extensive evaluations on English and multilingual language, and reasoning tasks, we demonstrate that PaLM 2 has significantly improved quality on downstream tasks across different model sizes, while simultaneously exhibiting faster and more efficient inference compared to PaLM. This improved efficiency enables broader deployment while also allowing the model to respond faster, for a more natural pace of interaction. PaLM 2 demonstrates robust reasoning capabilities exemplified by large improvements over PaLM on BIG-Bench and other reasoning tasks. PaLM 2 exhibits stable performance on a suite of responsible AI evaluations, and enables inference-time control over toxicity without additional overhead or impact on other capabilities. Overall, PaLM 2 achieves state-of-the-art performance across a diverse set of tasks and capabilities. When discussing the PaLM 2 family, it is important to distinguish between pre-trained models (of various sizes), fine-tuned variants of these models, and the user-facing products that use these models. In particular, user-facing products typically include additional pre- and post-processing steps. Additionally, the underlying models may evolve over time. Therefore, one should not expect the performance of user-facing products to exactly match the results reported in this report.

We demonstrate the potential of few-shot translation systems, trained with unpaired language data, for both high and low-resource language pairs. We show that with only 5 examples of high-quality translation data shown at inference, a transformer decoder-only model trained solely with self-supervised learning, is able to match specialized supervised state-of-the-art models as well as more general commercial translation systems. In particular, we outperform the best performing system on the WMT'21 English - Chinese news translation task by only using five examples of English - Chinese parallel data at inference. Moreover, our approach in building these models does not necessitate joint multilingual training or back-translation, is conceptually simple and shows the potential to extend to the multilingual setting. Furthermore, the resulting models are two orders of magnitude smaller than state-of-the-art language models. We then analyze the factors which impact the performance of few-shot translation systems, and highlight that the quality of the few-shot demonstrations heavily determines the quality of the translations generated by our models. Finally, we show that the few-shot paradigm also provides a way to control certain attributes of the translation -- we show that we are able to control for regional varieties and formality using only a five examples at inference, paving the way towards controllable machine translation systems.

We demonstrate that the absence of stable quasiparticle excitations on parts of the Fermi surface, similar to the "nodal-antinodal dichotomy" in underdoped cuprate superconductors, can be reproduced in models of strongly correlated electrons defined via a holographic dual. We show analytically that the anisotropy of the quantum critical continuum, which is a feature of these models, may lead to washing out the quasiparticle peak in one direction while leaving it intact in the perpendicular one. The effect relies on the qualitatively different scaling of the self-energy in different directions. Using the explicit example of the anisotropic Q-lattice model, we demonstrate how this effect emerges due to specific features of the near horizon geometry of the black hole in the dual description., Comment: 13 pages + appendices. 22 figure files. v2 with expanded references. v3 with minor revisions

In this paper we share findings from our effort to build practical machine translation (MT) systems capable of translating across over one thousand languages. We describe results in three research domains: (i) Building clean, web-mined datasets for 1500+ languages by leveraging semi-supervised pre-training for language identification and developing data-driven filtering techniques; (ii) Developing practical MT models for under-served languages by leveraging massively multilingual models trained with supervised parallel data for over 100 high-resource languages and monolingual datasets for an additional 1000+ languages; and (iii) Studying the limitations of evaluation metrics for these languages and conducting qualitative analysis of the outputs from our MT models, highlighting several frequent error modes of these types of models. We hope that our work provides useful insights to practitioners working towards building MT systems for currently understudied languages, and highlights research directions that can complement the weaknesses of massively multilingual models in data-sparse settings., Comment: V2: updated with some details from 24-language Google Translate launch in May 2022 V3: spelling corrections, additional acknowledgements

Quantum strongly correlated matter exhibits properties which are not easily explainable in the conventional framework of Fermi liquids. Universal effective field theory tools are applicable in these cases regardless of the microscopic details of the quantum system, since they are based on symmetries. It is necessary, however, to construct these effective tools in full generality, avoiding restrictions coming from particular microscopic descriptions which may inadequately constrain the coefficients that enter in the effective theory. In this work we demonstrate on explicit examples how the novel hydrodynamic coefficients which have been recently reinstated in the effective theory of pinned charge density waves (CDW) can affect the phenomenology of the thermo-electric transport in strongly correlated quantum matter. Our examples, based on two classes of holographic models with pinned CDW, have microscopics which are conceptually different from Fermi liquids. Therefore, the above novel transport coefficients are nonzero, contrary to the conventional approach. We show how these coefficients allow to take into account the change of sign of the Seebeck coefficient and the low resistivity of the CDW phase of the cuprate high temperature superconductors, without referring to the effects of Fermi surface reconstruction., Comment: 6 pages + supplementary material (v2: published version)

In this work, we study the effect of varying the architecture and training data quality on the data scaling properties of Neural Machine Translation (NMT). First, we establish that the test loss of encoder-decoder transformer models scales as a power law in the number of training samples, with a dependence on the model size. Then, we systematically vary aspects of the training setup to understand how they impact the data scaling laws. In particular, we change the following (1) Architecture and task setup: We compare to a transformer-LSTM hybrid, and a decoder-only transformer with a language modeling loss (2) Noise level in the training distribution: We experiment with filtering, and adding iid synthetic noise. In all the above cases, we find that the data scaling exponents are minimally impacted, suggesting that marginally worse architectures or training data can be compensated for by adding more data. Lastly, we find that using back-translated data instead of parallel data, can significantly degrade the scaling exponent.

We present LaMDA: Language Models for Dialog Applications. LaMDA is a family of Transformer-based neural language models specialized for dialog, which have up to 137B parameters and are pre-trained on 1.56T words of public dialog data and web text. While model scaling alone can improve quality, it shows less improvements on safety and factual grounding. We demonstrate that fine-tuning with annotated data and enabling the model to consult external knowledge sources can lead to significant improvements towards the two key challenges of safety and factual grounding. The first challenge, safety, involves ensuring that the model's responses are consistent with a set of human values, such as preventing harmful suggestions and unfair bias. We quantify safety using a metric based on an illustrative set of human values, and we find that filtering candidate responses using a LaMDA classifier fine-tuned with a small amount of crowdworker-annotated data offers a promising approach to improving model safety. The second challenge, factual grounding, involves enabling the model to consult external knowledge sources, such as an information retrieval system, a language translator, and a calculator. We quantify factuality using a groundedness metric, and we find that our approach enables the model to generate responses grounded in known sources, rather than responses that merely sound plausible. Finally, we explore the use of LaMDA in the domains of education and content recommendations, and analyze their helpfulness and role consistency.

Scaling language models with more data, compute and parameters has driven significant progress in natural language processing. For example, thanks to scaling, GPT-3 was able to achieve strong results on in-context learning tasks. However, training these large dense models requires significant amounts of computing resources. In this paper, we propose and develop a family of language models named GLaM (Generalist Language Model), which uses a sparsely activated mixture-of-experts architecture to scale the model capacity while also incurring substantially less training cost compared to dense variants. The largest GLaM has 1.2 trillion parameters, which is approximately 7x larger than GPT-3. It consumes only 1/3 of the energy used to train GPT-3 and requires half of the computation flops for inference, while still achieving better overall zero-shot and one-shot performance across 29 NLP tasks., Comment: Accepted to ICML 2022

The anomalous strange metal phase found in high-$T_c$ cuprates does not follow the conventional condensed-matter principles enshrined in the Fermi liquid and presents a great challenge for theory. Highly precise experimental determination of the electronic self-energy can provide a test bed for theoretical models of strange metals, and angle-resolved photoemission can provide this as a function of frequency, momentum, temperature and doping. Here we show that constant energy cuts through the nodal spectral function in (Pb,Bi)$_{2}$Sr$_{2-x}$La$_x$CuO$_{6+\delta}$ have a non-Lorentzian lineshape, meaning the nodal self-energy is $k$ dependent. We show that the experimental data are captured remarkably well by a power law with a $k$-dependent scaling exponent smoothly evolving with doping, a description that emerges naturally from AdS/CFT-based semi-holography. This puts a spotlight on holographic methods for the quantitative modelling of strongly interacting quantum materials like the cuprate strange metals.

Sparse Mixture-of-Experts (MoE) has been a successful approach for scaling multilingual translation models to billions of parameters without a proportional increase in training computation. However, MoE models are prohibitively large and practitioners often resort to methods such as distillation for serving. In this work, we investigate routing strategies at different granularity (token, sentence, task) in MoE models to bypass distillation. Experiments on WMT and a web-scale dataset suggest that task-level routing (task-MoE) enables us to extract smaller, ready-to-deploy sub-networks from large sparse models. On WMT, our task-MoE with 32 experts (533M parameters) outperforms the best performing token-level MoE model (token-MoE) by +1.0 BLEU on average across 30 language pairs. The peak inference throughput is also improved by a factor of 1.9x when we route by tasks instead of tokens. While distilling a token-MoE to a smaller dense model preserves only 32% of the BLEU gains, our sub-network task-MoE, by design, preserves all the gains with the same inference cost as the distilled student model. Finally, when scaling up to 200 language pairs, our 128-expert task-MoE (13B parameters) performs competitively with a token-level counterpart, while improving the peak inference throughput by a factor of 2.6x., Comment: EMNLP Findings 2021

We present an empirical study of scaling properties of encoder-decoder Transformer models used in neural machine translation (NMT). We show that cross-entropy loss as a function of model size follows a certain scaling law. Specifically (i) We propose a formula which describes the scaling behavior of cross-entropy loss as a bivariate function of encoder and decoder size, and show that it gives accurate predictions under a variety of scaling approaches and languages; we show that the total number of parameters alone is not sufficient for such purposes. (ii) We observe different power law exponents when scaling the decoder vs scaling the encoder, and provide recommendations for optimal allocation of encoder/decoder capacity based on this observation. (iii) We also report that the scaling behavior of the model is acutely influenced by composition bias of the train/test sets, which we define as any deviation from naturally generated text (either via machine generated or human translated text). We observe that natural text on the target side enjoys scaling, which manifests as successful reduction of the cross-entropy loss. (iv) Finally, we investigate the relationship between the cross-entropy loss and the quality of the generated translations. We find two different behaviors, depending on the nature of the test data. For test sets which were originally translated from target language to source language, both loss and BLEU score improve as model size increases. In contrast, for test sets originally translated from source language to target language, the loss improves, but the BLEU score stops improving after a certain threshold. We release generated text from all models used in this study., Comment: 31 pages, 23 figures

We present GSPMD, an automatic, compiler-based parallelization system for common machine learning computations. It allows users to write programs in the same way as for a single device, then give hints through a few annotations on how to distribute tensors, based on which GSPMD will parallelize the computation. Its representation of partitioning is simple yet general, allowing it to express different or mixed paradigms of parallelism on a wide variety of models. GSPMD infers the partitioning for every operator based on limited user annotations, making it convenient to scale existing single-device programs. It solves several technical challenges for production usage, allowing GSPMD to achieve 50% to 62% compute utilization on up to 2048 Cloud TPUv3 cores for models with up to one trillion parameters.

We study the dynamics of spontaneous translation symmetry breaking in holographic models in presence of weak explicit sources. We show that, unlike conventional gapped quantum charge density wave systems, this dynamics is well characterized by the effective time dependent Ginzburg-Landau equation, both above and below the critical temperature, which leads to a "gapless" algebraic pattern of metal-insulator phase transition. In this framework we elucidate the nature of the damped Goldstone mode (the phason), which has earlier been identified in the effective hydrodynamic theory of pinned charge density wave and observed in holographic homogeneous lattice models. We follow the motion of the quasinormal modes across the dynamical phase transition in models with either periodic inhomogeneous or helical homogeneous spatial structures, showing that the phase relaxation rate is continuous at the critical temperature. Moreover, we find that the qualitative low-energy dynamics of the broken phase is universal, insensitive to the precise pattern of translation symmetry breaking, and therefore applies to homogeneous models as well., Comment: 25 pages + appendices, 13 figures

Neural network scaling has been critical for improving the model quality in many real-world machine learning applications with vast amounts of training data and compute. Although this trend of scaling is affirmed to be a sure-fire approach for better model quality, there are challenges on the path such as the computation cost, ease of programming, and efficient implementation on parallel devices. GShard is a module composed of a set of lightweight annotation APIs and an extension to the XLA compiler. It provides an elegant way to express a wide range of parallel computation patterns with minimal changes to the existing model code. GShard enabled us to scale up multilingual neural machine translation Transformer model with Sparsely-Gated Mixture-of-Experts beyond 600 billion parameters using automatic sharding. We demonstrate that such a giant model can efficiently be trained on 2048 TPU v3 accelerators in 4 days to achieve far superior quality for translation from 100 languages to English compared to the prior art.

We analyze fermionic response of strongly correlated holographic matter in presence of inhomogeneous periodically modulated potential mimicking the crystal lattice. The modulation is sourced by a scalar operator that explicitly breaks the translational symmetry in one direction. We compute the fermion spectral function and show that it either exhibits a well defined Fermi surface with umklapp gaps opening on the Brillouin zone boundary at small lattice wave vector, or, when the wave vector is large, the Fermi surface is anisotropically deformed and the quasiparticles get significantly broadened in the direction of translation symmetry breaking. Making use of the ability of our model to smoothly extrapolate to the homogeneous Q-lattice like setup, we show that this novel effect is not due to the periodic modulation of the potential and Umklapp physics, but rather due to the anisotropic features of the holographic horizon. That means it encodes novel physics of strongly correlated critical systems which may be relevant for phenomenology of exotic states of electron matter., Comment: 25 pages, 17 figures

We study the charge density response in holographic models with explicit translation symmetry breaking which is relevant in IR. In particular, we focus on Q-lattices and the Bianchy VII helix. We show that the hydrodynamic sound mode is removed from the spectrum due to the strong momentum relaxation and therefore, the usual treatment of the plasmon as Coulomb-dressed zero sound does not apply. Furthermore, the dominant coherent modes in the longitudinal channel, which control the neutral density-density correlator, are diffusive. We show these modes are strongly suppressed when the boundary Coulomb interaction is turned on. This renders the low frequency charge density response spectrum completely incoherent and featureless. At intermediate frequencies, we observe a broad feature -- the fake plasmon -- in the dressed correlator, which could be confused with an overdamped plasmon. However, its gap is set by the scale of translation symmetry breaking instead of the plasma frequency. This broad feature originates from the non-hydrodynamic sector of the holographic spectrum, and therefore, its behaviour, typical of strongly correlated quantum critical systems with holographic duals, deviates from the standard Fermi-liquid paradigm., Comment: Published version, fake plasmons in insulating Q-lattice and Helix

We study the fermionic spectral density in a strongly correlated quantum system described by a gravity dual. In the presence of periodically modulated chemical potential, which models the effect of the ionic lattice, we explore the shapes of the corresponding Fermi surfaces, defined by the location of peaks in the spectral density at the Fermi level. We find that at strong lattice potentials sectors of the Fermi surface are unexpectedly destroyed and the Fermi surface becomes an arc-like disconnected manifold. We explain this phenomenon in terms of a collision of the Fermi surface pole with zeros of the fermionic Green's function, which are explicitly computable in the holographic dual., Comment: 24 pages, 10 figures, plus appendices; v2: references added

We introduce our efforts towards building a universal neural machine translation (NMT) system capable of translating between any language pair. We set a milestone towards this goal by building a single massively multilingual NMT model handling 103 languages trained on over 25 billion examples. Our system demonstrates effective transfer learning ability, significantly improving translation quality of low-resource languages, while keeping high-resource language translation quality on-par with competitive bilingual baselines. We provide in-depth analysis of various aspects of model building that are crucial to achieving quality and practicality in universal NMT. While we prototype a high-quality universal translation system, our extensive empirical analysis exposes issues that need to be further addressed, and we suggest directions for future research.

In this work we study the relaxation of the system of strongly correlated electrons, at charge neutrality, when the chemical potential undergoes a local change. This setup is a model for for the X-ray absorbtion edge study in the half-filled graphene. We use holographic duality to describe the system as a classical Schwarzschild black hole in curved 4-dimensional AdS spacetime. Assuming the amplitude of the quench is small, we neglect the backreaction on the geometry. We numerically study the two relaxation regimes: the adiabatic relaxation when the quench is slow and the relaxation governed by the quasinormal modes of the system, when the quench is fast. We confirm the expectation that the scale of separation between the slow and fast regimes is set by the characteristic frequency of the quasinormal modes., Comment: 11 pages, 5 figures; v2: references updated, treatment of scales improved; v3: technical mistakes corrected, treatment of QNMs improved; v4: treatment of evolution equation improved, Bessel analysis added, journal accepted version

We consider the problem of making efficient use of heterogeneous training data in neural machine translation (NMT). Specifically, given a training dataset with a sentence-level feature such as noise, we seek an optimal curriculum, or order for presenting examples to the system during training. Our curriculum framework allows examples to appear an arbitrary number of times, and thus generalizes data weighting, filtering, and fine-tuning schemes. Rather than relying on prior knowledge to design a curriculum, we use reinforcement learning to learn one automatically, jointly with the NMT system, in the course of a single training run. We show that this approach can beat uniform and filtering baselines on Paracrawl and WMT English-to-French datasets by up to +3.4 BLEU, and match the performance of a hand-designed, state-of-the-art curriculum., Comment: NAACL 2019 short paper. Reviewer comments not yet addressed

Lingvo is a Tensorflow framework offering a complete solution for collaborative deep learning research, with a particular focus towards sequence-to-sequence models. Lingvo models are composed of modular building blocks that are flexible and easily extensible, and experiment configurations are centralized and highly customizable. Distributed training and quantized inference are supported directly within the framework, and it contains existing implementations of a large number of utilities, helper functions, and the newest research ideas. Lingvo has been used in collaboration by dozens of researchers in more than 20 papers over the last two years. This document outlines the underlying design of Lingvo and serves as an introduction to the various pieces of the framework, while also offering examples of advanced features that showcase the capabilities of the framework.

Holographic strange metals are known to have a power law resistivity rising with temperature, which is reminiscent of the strange metal phases in condensed matter systems. In some holographic models, however, the exponent of the power law in the resistivity can be negative. In this case one encounters phases with diverging resistivity at zero temperature: holographic strange insulators. These states arise as a result of translational symmetry breaking in the system, which can either be strong explicit and relevant in the IR, or spontaneous, but pinned by a small explicit source. In some regards, one can associate these two classes to the normal band insulators due to the strong ionic potential, and Mott insulator due to the commensurate lock in of the charge density wave. We study different features of these classes on the explicit example of a holographic helical model with homogeneous Bianchy VII type translational symmetry breaking, and uncover the main mechanisms underlying transport in these two cases. We find that while transport in the explicit relevant case is governed by the incoherent conductivity, in the pinned spontaneous case the leading contribution comes from the coherent part., Comment: 21 pages + appendices, 15 figures; v2: revised version

The plasmon is a ubiquitous collective mode in charged liquids. Due to the long-range Coulomb interaction, the massless zero sound mode of the neutral system acquires a finite plasmon frequency in the long-wavelength limit. In the zero-temperature state of conventional metals -- the Fermi liquid -- the plasmon lives infinitely long at long wavelength when the system is (effectively) translationally invariant. In contrast, we will show that in strongly entangled strange metals the protection of zero sound fails at finite frequency and plasmons are always short lived regardless of their wavelength. Computing the explicit plasmon response in holographic strange metals as an example, we show that decay into the quantum critical continuum replaces Landau damping and this happens for any wavelength., Comment: Published version -- Editor's Suggestions

Excess heat in organic light emitting diodes (OLEDs) that is produced during their operation may accelerate their degradation and may cause an inhomogeneous brightness distribution, in particular in large area OLEDs. Assessing the quantitative impact of heat excess is difficult, because all decisive processes related to charge transport and emission via charge recombination are thermally activated. For example, electric currents that are elevated due to larger temperatures cause additional Joule heating and increase, hence, the device temperature even further. Here we establish how parameters responsible for heat transport, i.e., the thermal conductivity of the organic layers and the heat transfer coefficient between the device surface and the environment govern the temperature inside the OLED. We establish, relying on three-dimensional drift-diffusion simulations that self-consistently couple thermally-activated charge transport and heat transport, that the thermal conductivity of organic layers is not a bottleneck for heat transport, because the encountered layer thicknesses in realistic device geometries prevent heat accumulation. The heat transfer to the ambient environment is the key parameter to dissipate excess heat from the device. Intentionally elevated operating temperatures, that may improve the OLEDs electric performance, are not necessarily beneficial, as any increase in operating temperature decreases the device stability. The thermal effects being decisive for the OLED temperature occur in device layers beyond the electrically active region. We propose analytical expressions that relate the temperature in the device for a given point of operation to the heat transfer to the environment and the substrate., Comment: 19 pages, 5 figures

We study the observable properties of quantum systems which involve a quantum continuum as a subpart. We show in a very general way that in any system, which consists of at least two isolated states coupled to a continuum, the spectral function of one of the states exhibits an isolated zero at the energy of the other state. Several examples of quantum systems exhibiting such isolated zeros are discussed. Although very general, this phenomenon can be particularly useful as an indirect detection tool for the continuum spectrum in the lab realizations of quantum critical behavior., Comment: 10 pages, 6 figures; Published version

These are the notes for a series of Numerical Study group meetings, held in Lorentz institute in the fall of 2017. The aim of the notes is to provide a non-specialist with the minimal knowledge in numerical methods used in BVP for PDEs, necessary to solve the problems typically arising in applications of holography to condensed matter systems. A graduate level knowledge of Linear Algebra and theory of Differential Equations is assumed. Special attention is payed to the treatment of the boundary conditions of general form. The notes focus on the practical aspects of the implementation leaving aside the theory behind the methods in use. A few simple problems to test the acquired knowledge are included., Comment: 23 pages, Lecture notes

When the system with internal tendency to a spontaneous formation of a spatially periodic state is brought in contact with the external explicit periodic potential, the interesting phenomenon of commensurate lock in can be observed. In case when the explicit potential is strong enough and its period is close to the period of the spontaneous structure, the latter is forced to assume the periodicity of the former and the commensurate state becomes a thermodynamically preferred one. If instead the two periods are significantly different, the incommensurate state is formed. It is characterized by a finite density of solitonic objects -- discommensurations -- on top of the commensurate state. In this note I study the properties of discommensurations in holographic model with inhomogeneous translational symmetry breaking and explain how one can understand the commensurate/incommensurate phase transition as a proliferation of these solitons. Some useful numerical techniques are discussed in the Appendix., Comment: 22 pages, 9 figures. v2: misprints corrected, reverences added

Mott insulators form because of strong electron repulsions, being at the heart of strongly correlated electron physics. Conventionally these are understood as classical "traffic jams" of electrons described by a short-ranged entangled product ground state. Exploiting the holographic duality, which maps the physics of densely entangled matter onto gravitational black hole physics, we show how Mott-insulators can be constructed departing from entangled non-Fermi liquid metallic states, such as the strange metals found in cuprate superconductors. These "entangled Mott insulators" have traits in common with the "classical" Mott insulators, such as the formation of Mott gap in the optical conductivity, super-exchange-like interactions, and form "stripes" when doped. They also exhibit new properties: the ordering wave vectors are detached from the number of electrons in the unit cell, and the DC resistivity diverges algebraically instead of exponentially as function of temperature. These results may shed light on the mysterious ordering phenomena observed in underdoped cuprates., Comment: 27 pages, 9 figures. Accepted in Nature Physics

We consider the spontaneous breaking of translational symmetry and identify the associated Goldstone mode -- a longitudinal phonon -- in a holographic model with Bianchi VII helical symmetry. For the first time in holography, we observe the pinning of this mode after introducing a source for explicit breaking compatible with the helical symmetry of our setup. We study the dispersion relation of the resulting pseudo-Goldstone mode, uncovering how its speed and mass gap depend on the amplitude of the source and temperature. In addition, we extract the optical conductivity as a function of frequency, which reveals a metal-insulator transition as a consequence of the pinning., Comment: 24 pages, 14 figures. v2: comments and references added; v3: discussions added, slight change of title, version published in JHEP

We consider the spontaneous formation of striped structures in a holographic model which possesses explicit translational symmetry breaking, dual to an ionic lattice with spatially modulated chemical potential. We focus on the perturbative study of the marginal modes which drive the transition to a phase exhibiting spontaneous stripes. We study the wave-vectors of the instabilities with largest critical temperature in a wide range of backgrounds characterized by the period and the amplitude of the chemical potential modulation. We report the first holographic observation of the commensurate lock-in between the spontaneous stripes and the underlying ionic lattice, which takes place when the amplitude of the lattice is large enough. We also observe an incommensurate regime in which the amplitude of the lattice is finite, but the preferred stripe wave-vector is different from that of the lattice., Comment: 21 page, 10 figures; v2: references added, v3: published version

We propose a simple solution to use a single Neural Machine Translation (NMT) model to translate between multiple languages. Our solution requires no change in the model architecture from our base system but instead introduces an artificial token at the beginning of the input sentence to specify the required target language. The rest of the model, which includes encoder, decoder and attention, remains unchanged and is shared across all languages. Using a shared wordpiece vocabulary, our approach enables Multilingual NMT using a single model without any increase in parameters, which is significantly simpler than previous proposals for Multilingual NMT. Our method often improves the translation quality of all involved language pairs, even while keeping the total number of model parameters constant. On the WMT'14 benchmarks, a single multilingual model achieves comparable performance for English$\rightarrow$French and surpasses state-of-the-art results for English$\rightarrow$German. Similarly, a single multilingual model surpasses state-of-the-art results for French$\rightarrow$English and German$\rightarrow$English on WMT'14 and WMT'15 benchmarks respectively. On production corpora, multilingual models of up to twelve language pairs allow for better translation of many individual pairs. In addition to improving the translation quality of language pairs that the model was trained with, our models can also learn to perform implicit bridging between language pairs never seen explicitly during training, showing that transfer learning and zero-shot translation is possible for neural translation. Finally, we show analyses that hints at a universal interlingua representation in our models and show some interesting examples when mixing languages.

Neural Machine Translation (NMT) is an end-to-end learning approach for automated translation, with the potential to overcome many of the weaknesses of conventional phrase-based translation systems. Unfortunately, NMT systems are known to be computationally expensive both in training and in translation inference. Also, most NMT systems have difficulty with rare words. These issues have hindered NMT's use in practical deployments and services, where both accuracy and speed are essential. In this work, we present GNMT, Google's Neural Machine Translation system, which attempts to address many of these issues. Our model consists of a deep LSTM network with 8 encoder and 8 decoder layers using attention and residual connections. To improve parallelism and therefore decrease training time, our attention mechanism connects the bottom layer of the decoder to the top layer of the encoder. To accelerate the final translation speed, we employ low-precision arithmetic during inference computations. To improve handling of rare words, we divide words into a limited set of common sub-word units ("wordpieces") for both input and output. This method provides a good balance between the flexibility of "character"-delimited models and the efficiency of "word"-delimited models, naturally handles translation of rare words, and ultimately improves the overall accuracy of the system. Our beam search technique employs a length-normalization procedure and uses a coverage penalty, which encourages generation of an output sentence that is most likely to cover all the words in the source sentence. On the WMT'14 English-to-French and English-to-German benchmarks, GNMT achieves competitive results to state-of-the-art. Using a human side-by-side evaluation on a set of isolated simple sentences, it reduces translation errors by an average of 60% compared to Google's phrase-based production system.

It is presently unknown how strong lattice potentials influence the fermion spectral function of the holographic strange metals predicted by the AdS/CFT correspondence. This embodies a crucial test for the application of holography to condensed matter experiments. We show that for one particular momentum direction this spectrum can be computed for arbitrary strength of the effective translational symmetry breaking potential of the so-called Bianchi-VII geometry employing ordinary differential equations. Deep in the strange metal regime we find rather small changes to the single-fermion response computed by the emergent quantum critical IR, even when the potential becomes relevant in the infra-red. However, in the regime where holographic quasi-particles occur, defining a Fermi surface in the continuum, they acquire a finite lifetime at any finite potential strength. At the transition from irrelevancy to relevancy of the Bianchi potential in the deep infra-red the quasi-particle remnants disappear completely and the fermion spectrum exhibits a purely relaxational behaviour., Comment: 30 pages, 10 figures

An interesting application of the gauge/gravity duality to condensed matter physics is the description of a lattice via breaking translational invariance on the gravity side. By making use of global symmetries, it is possible to do so without scarifying homogeneity of the pertinent bulk solutions, which we thus term as "homogeneous holographic lattices." Due to their technical simplicity, these configurations have received a great deal of attention in the last few years and have been shown to correctly describe momentum relaxation and hence (finite) DC conductivities. However, it is not clear whether they are able to capture other lattice effects which are of interest in condensed matter. In this paper we investigate this question focusing our attention on the phenomenon of commensurability, which arises when the lattice scale is tuned to be equal to (an integer multiple of) another momentum scale in the system. We do so by studying the formation of spatially modulated phases in various models of homogeneous holographic lattices. Our results indicate that the onset of the instability is controlled by the near horizon geometry, which for insulating solutions does carry information about the lattice. However, we observe no sharp connection between the characteristic momentum of the broken phase and the lattice pitch, which calls into question the applicability of these models to the physics of commensurability., Comment: 23 pages, 10 figures; v2: references added

We study different phases in the holographic model of d-wave superconductor. These are described by solutions to the classical equations of motion found in different ansatze. Apart from the known homogeneous d-wave superconducting phase we find three new solutions. Two of them represent two distinct families of the spatially modulated solutions, which realize the charge density wave phases in the dual theory. The third one is the new homogeneous phase with nonzero anapole moment. These phases are relevant to the physics of cuprate high-Tc superconductor in pseudogap region. While the d-wave phase preserves translation, parity and time reversal symmetry, the striped phases break translations spontaneously. Parity and time-reversal are preserved when combined with discrete half-periodic shift of the wave. In anapole phase translation symmetry is preserved, but parity and time reversal are spontaneously broken. All of the considered solutions brake the global $U(1)$. Thermodynamical treatment shows that in the simplest d-wave model the anapole phase is always preferred, while the stripe phases realize the continuous transition in solution space between the normal phase and two homogeneous condensed phases., Comment: 18 pages, 7 figures

We study the relationship between chiral symmetry breaking and baryons in holographic QCD. We construct a soliton with unit baryon charge in the presence of a nonzero mean value of the scalar bifundamental field, which is dual to the chiral condensate. We obtain a relation between the chiral condensate and the mass of the baryon and find in a clear-cut way that at large values of the condensate the holographic soliton is no longer located on the IR wall. Instead it is split into two halves, which are symmetrically located on the left and right flavor branes. On the other hand we find that the local value of the quark condensate is suppressed in the core of the soliton, which is evidence for a partial chiral symmetry restoration inside the baryon., Comment: 26 pages, 8 figures

We study local temperature fluctuations in a 2+1 dimensional CFT on the sphere, dual to a black hole in asymptotically AdS spacetime. The fluctuation spectrum is governed by the lowest-lying hydrodynamic modes of the system whose frequency and damping rate determine whether temperature fluctuations are thermal or quantum. We calculate numerically the corresponding quasinormal frequencies and match the result with the hydrodynamics of the dual CFT at large temperature. As a by-product of our analysis we determine the appropriate boundary conditions for calculating low-lying quasinormal modes for a four-dimensional Reissner-Nordstr\"om black hole in global AdS., Comment: LaTeX: 31 pages, 7 figures; V2: reference added; V3: added/updated charged case

We briefly review the generalized Skyrmion model for the baryon recently suggested by us. It takes into account the tower of vector and axial mesons as well as the chiral symmetry breaking. The generalized Skyrmion model provides the qualitative explanation of the Ioffe's formula for the baryon mass., Comment: 7 pages, Contribution to the Proceedings of the II Russian-Spanish Congress "Particle and Nuclear Physics at all Scales and Cosmology", Saint-Petersburg, October 1-4, 2013

We report the observation of the spatially modulated static mode in the spectrum of fluctuations around the condensed phase of the holographic d-wave superconductor. The mode involves the time component of the gauge field that is related to the charge density wave in the dual superconductor. No additional ingredients are added to the action of five dimensional theory. We speculate on the relevance of the observed mode to the formation of the pseudogap state., Comment: jheppub, 17 pages, 4 figures. v2: temperature normalization fixed, v3: the journal version, discussion of P-symmetry improved

We discuss the description of baryon as the dyonic instanton in holographic QCD. The solution generalizes the Skyrmion taking into account the infinite tower of vector and axial mesons as well as the chiral condensate. We construct the solution with unit baryon charge and study the dependence of its mass on the chiral condensate. The elegant explanation of the Ioffe's formula has been found and we speculate on the relation between physical scales of the chiral and conformal symmetry breaking., Comment: 24 pages, 2 figures, 2 tables. v.2: references added

Stochastic modeling of phylogenies raises five questions that have received varying levels of attention from quantitatively inclined biologists. 1) How large do we expect (from the model) the ration of maximum historical diversity to current diversity to be? 2) From a correct phylogeny of the extant species of a clade, what can we deduce about past speciation and extinction rates? 3) What proportion of extant species are in fact descendants of still-extant ancestral species, and how does this compare with predictions od models? 4) When one moves from trees on species to trees on sets of species (whether traditional higher order taxa or clades from PhyloCode), does one expect trees to become more unbiased as a purely logical consequence of tree structure, without signifying any real biological phenomenon? 5) How do we expect that fluctuation rates for counts of higher order taxa should compare with fluctuation rates for number of species? WE present a mathematician's view based on an oversimplified modeling framework in which all these questions can be studied coherently., Comment: 11 pages, 3 figures

We construct the holographic model of an $S^\pm$ multiband superconductor. This system is a candidate to explain the anomalous features of the iron-based superconductors (e.g. LaFeAsO, BFe2As2, and other pnictides and arsenides). We study the framework, which allows formation of the sign-interchanging order parameter. We also calculate the electric AC conductivity and study its features, related to the interband interaction., Comment: 18 pages, 5 figures. v2: references added, misprints corrected. v3: published version

The main substance of the paper concerns the growth rate and the classification (ergodicity, transience) of a family of random trees. In the basic model, new edges appear according to a Poisson process of parameter $\lambda$ and leaves can be deleted at a rate $\mu$. The main results lay the stress on the famous number $e$. A complete classification of the process is given in terms of the intensity factor $\rho=\lambda/\mu $: it is ergodic if $\rho\leq e^{-1}$, and transient if $\rho>e^{-1}$. There is a phase transition phenomenon: the usual region of null recurrence (in the parameter space) here does not exist. This fact is rare for countable Markov chains with exponentially distributed jumps. Some basic stationary laws are computed, e.g. the number of vertices and the height. Various bounds, limit laws and ergodic-like theorems are obtained, both for the transient and ergodic regimes. In particular, when the system is transient, the height of the tree grows linearly as the time $t\to\infty$, at a rate which is explicitly computed. Some of the results are extended to the so-called multiclass model.

We discuss the baryon in the holographic QCD framework and focus on the role of the bifundamental scalar field, realizing chiral symmetry breaking. We suggest the interpretation of a baryon as a dyonic instanton within the Atiyah-Manton-like approach for the flavor gauge group in the peculiar cylindrical ansatz in five-dimensional theory. Our approach provides the new mechanism of the stabilization of the baryon size., Comment: 19 pages, 2 figures, v2: version to be published in PRD. Introduction improved, references added, typos corrected

In this Letter we discuss a few issues concerning the magnetic susceptibility of the quark condensate and the Son-Yamamoto (SY) anomaly matching equation. It is shown that the SY relation in the IR implies a nontrivial interplay between the kinetic and WZW terms in the chiral Lagrangian. It is also demonstrated that in a holographic framework an external magnetic field triggers mixing between scalar and tensor fields. Accounting for this, one may calculate the magnetic susceptibility of the quark condensate to all orders in the magnetic field., Comment: 20 pages, 2 figures

The aim of this work is to study the holographic dual to the gauge theory with a nonzero gluon condensate. We check for consistency the holographic way of describing the condensate and calculate the expectation value of a small Wilson loop in the presence of the gluon condensate, thus obtaining the relevant coefficient in the operator product expansion of the small loop in different holographic models. We also study the effect of the condensate on the Gross-Ooguri phase transition in the correlator of two circular Wilson loops in parallel and concentric configurations. In the numerical study of the concentric case, we find that the phase transition changes its order when the size of the loops is of order of the gluon condensate. We report this change of the phase transition order to be a new effect in Wilson loop correlators., Comment: 24 pages, 6 figures

In the context of gauge/gravity duality, we verify two types of gauge theory low-energy theorems, the dilation Ward identities and the decoupling of heavy flavor. First, we provide an analytic proof of non-trivial dilation Ward identities for a theory holographically dual to a background with gluon condensate (the self-dual Liu--Tseytlin background). In this way an important class of low-energy theorems for correlators of different operators with the trace of the energy-momentum tensor is established, which so far has been studied in field theory only. Another low-energy relationship, the so-called decoupling theorem, is numerically shown to hold universally in three holographic models involving both the quark and the gluon condensate. We show this by comparing the ratio of the quark and gluon condensates in three different examples of gravity backgrounds with non-trivial dilaton flow. As a by-product of our study, we also obtain gauge field condensate contributions to meson transport coefficients., Comment: 32 pages, 4 figures, two references added, typos removed