Search

Transformer-based pre-trained language models have proven to be effective for learning contextualized language representation. However, current approaches only take advantage of the output of the encoder's final layer when fine-tuning the downstream tasks. We argue that only taking single layer's output restricts the power of pre-trained representation. Thus we deepen the representation learned by the model by fusing the hidden representation in terms of an explicit HIdden Representation Extractor (HIRE), which automatically absorbs the complementary representation with respect to the output from the final layer. Utilizing RoBERTa as the backbone encoder, our proposed improvement over the pre-trained models is shown effective on multiple natural language understanding tasks and help our model rival with the state-of-the-art models on the GLUE benchmark.

Elastic electron scattering has been used to paint the most accurate picture of the proton distribution in atomic nuclei. Spurred by new experimental developments, it is now possible to gain valuable insights into the neutron distribution using exclusively electroweak probes. Our goal is to assess the information content and complementarity of the following three electroweak experiments in constraining the neutron distribution of atomic nuclei: (a) parity violating elastic electron scattering, (b) coherent elastic neutrino nucleus scattering, and (c) elastic electron scattering of unstable nuclei. Relativistic mean-field models informed by the properties of finite nuclei and neutron stars are used to compute ground state densities and form factors of a variety of nuclei. All the models follow the same fitting protocol, except for the assumed and presently unknown value of the neutron skin thickness of 208Pb. This enables one to tune the density dependence of the symmetry energy without compromising the success in reproducing well known physical observables. We found that the ongoing PREX-II and upcoming CREX campaigns at Jefferson Lab will play a vital role in constraining the weak form factor of xenon and argon, liquid noble gases that are used for the detection of both neutrinos and dark matter particles. We concluded that remarkable new advances in experimental physics have opened a new window into ground state densities of atomic nuclei using solely electroweak probes. The diversity and versatility of these experiments reveal powerful correlations that impose important nuclear structure constraints. In turn, these constraints provide quantitative theoretical uncertainties that are instrumental in searches for new physics and insights into the behavior of dense matter., Comment: 18 pages, 11 figures

Stochastic gradient descent (SGD) has been found to be surprisingly effective in training a variety of deep neural networks. However, there is still a lack of understanding on how and why SGD can train these complex networks towards a global minimum. In this study, we establish the convergence of SGD to a global minimum for nonconvex optimization problems that are commonly encountered in neural network training. Our argument exploits the following two important properties: 1) the training loss can achieve zero value (approximately), which has been widely observed in deep learning; 2) SGD follows a star-convex path, which is verified by various experiments in this paper. In such a context, our analysis shows that SGD, although has long been considered as a randomized algorithm, converges in an intrinsically deterministic manner to a global minimum., Comment: ICLR2019

Electronic tunability in crystals with weakly-bound layers can be achieved through layer stacking order. One such example is MoTe$_2$, where the low-temperature orthorhombic T$_d$ phase is topological and host to Weyl quasiparticles. The transition mechanism to the non-trivial topology is elucidated by single crystal neutron diffraction. Upon cooling from the monoclinic 1T$\prime$ to the T$_d$ phase, diffuse scattering accompanies the transition, arising from random, in-plane layer displacements, and dissipates upon entering the T$_d$ phase. Diffuse scattering is observed only in the H0L plane due to irreversible layer shifts along the c-axis that break the centrosymmetry of the monoclinic lattice., Comment: This paper contains 8 pages including references and supplement materials. 3 figures and 1 table