Search

Auto-evaluating language models (LMs), i.e., using a grader LM to evaluate the candidate LM, is an appealing way to accelerate the evaluation process and the cost associated with it. But this presents a paradox: how can we trust the grader LM, which is presumably weaker than the candidate LM, to assess problems that are beyond the frontier of the capabilities of either model or both? For instance, today's LMs struggle on graduate-level physics and Olympiad-level math, making them unreliable graders in these domains. We show that providing privileged information -- such as ground-truth solutions or problem-specific guidelines -- improves automated evaluations on such frontier problems. This approach offers two key advantages. First, it expands the range of problems where LMs graders apply. Specifically, weaker models can now rate the predictions of stronger models. Second, privileged information can be used to devise easier variations of challenging problems which improves the separability of different LMs on tasks where their performance is generally low. With this approach, general-purpose LM graders match the state of the art performance on RewardBench, surpassing almost all the specially-tuned models. LM graders also outperform individual human raters on Vibe-Eval, and approach human expert graders on Olympiad-level math problems.

Compton-thick active galactic nuclei (CT-AGNs), characterized by a significant absorption with column densities of $\mathrm{N_H}\geqslant 1.5\times 10^{24} \ \mathrm{cm}^{-2}$, emit feeble X-ray radiation and are even undetectable by X-ray instruments, making them difficult to identify. X-ray radiation from AGNs is the predominant source of the cosmic X-ray background (CXB). Based on AGN synthesis models for the CXB, the fraction of CT-AGNs should constitute a substantial portion of AGN population, approximately 30\% or more. The fraction of CT-AGNs discovered in the Cosmological Evolution Survey (COSMOS) is significantly lower than this value. This means that many CT-AGNs may be hidden in AGNs that exhibit low photon counts or that have not been detected by X-ray instruments. This work focuses on identifying CT-AGNs hidden in AGNs with low photon counts. Firstly, we selected 440 AGNs with abundant multiwavelength data as our sample. Secondly, we analyzed multiwavelength data, extracting crucial physical parameters required for the CT-AGN diagnosis. Finally, we used multiwavelength approaches to identify CT-AGNs. We have successfully identified 18 CT-AGNs in our sample. Among the CT-AGNs, four AGNs show discrepant results across different diagnostic methods. We discuss the potential reasons behind these diagnostic discrepancies. We explore the impact of estimating [O~III]$\lambda~5007$ luminosities based on [O~II]$\lambda~3727$ luminosities for the CT-AGN diagnosis. We have also found that the properties of host galaxies for CT-AGNs and non-CT-AGNs do not show significant discrepancies., Comment: 12 pages, 7 figures, 4 tables. Accepted in Astronomy & Astrophysics

Studying the periodic flux-variation behavior of blazars is vital for probing supermassive black hole binaries and the kinematics of relativistic jets. In this work, we report the detection of the multi-band possible periodic variations of the blazar PKS J2134-0153, including the infrared ($1.6(\pm0.4)\times 10^3$ days) and optical ($1.8(\pm1)\times 10^3$ days). The periods in the infrared and optical bands are statistically consistent with the period in the radio band ($P_{\mathrm{Radio}}$$ = 1760\pm33$ days, obtained from our previous work). Moreover, flux variations in different bands are correlated with evident inter-band time delays, and the time lags of infrared and optical emission with respect to radio emission are $(3.3\pm2.3)\times10^{2}$ days and $(3.0\pm2.3)\times10^{2}$ days, respectively. The cross-correlations indicate a common origin of radio, infrared, and optical emission. The relative positions between emission regions of infrared and optical emission to radio emission are estimated according to the time lags, i.e., $0.37\pm0.26$ pc and $0.33\pm0.26$ pc. The relative distances seem to be quantitatively consistent with the theoretical prediction., Comment: Accepted to MNRAS

The theoretical model suggests that relativistic jets of AGN rely on the black hole spin and/or accretion. We study the relationship between jet, accretion, and spin using supermassive black hole samples with reliable spin of black holes. Our results are as follows: (1) There is a weak correlation between radio luminosity and the spin of black hole for our sample, which may imply that the jet of the supermassive black hole in our sample depends on the other physical parameters besides black hole spins, such as accretion disk luminosity. (2) The jet power of a supermassive black hole can be explained by the hybrid model with magnetic field of corona. (3) There is a significant correlation between radio-loudness and black hole spin for our sample. These sources with high radio-loudness tend to have high black hole spins. These results provide observational evidence that the black hole spin may explain the bimodal phenomena of radio-loud and radio-quiet AGN., Comment: 13pages,4figures, accept for publication in RAA

In this paper, we propose a novel minimal physical model to elucidate the long-term stochastic variability of blazars. The model is built on the realistic background of magnetized plasma jets dissipating energy through a turbulent cascade process that transfers energy to small-scale structures with highly anisotropic radiation. The model demonstrates the ability to spontaneously generate variability features consistent with observations of blazars under uniformly random fluctuations in the underlying physical parameters. This indicates that the model possesses self-similarity across multiple time scales, providing a natural explanation for the universal power spectral density (PSD) structure observed in different types of blazars. Moreover, the model exhibits that when the cascade process produces a relatively flat blob energy distribution, the spectral index of the model-simulated PSD in the high-frequency regime will be steeper than that predicted by the Damped Random Walk (DRW) model, which is in agreement with recent observations of active galactic nucleus (AGN) variability, providing a plausible theoretical explanation. The model is also able to reproduce the observed fractional variability amplitude (FVA) characteristics of blazars, and suggests that the specific particle acceleration and radiative cooling processes within the blob may not be the key factor shaping the long-term stochastic variability. This minimal model provides a new physical perspective for understanding the long-term stochastic variability of blazars., Comment: 12 pages, 3 figures, accepted for publication in PRD

We present here the first systematic search of short timescale $\gamma$-ray flares from 29 high Galactic latitude BL Lac objects over 14 years of Fermi Large Area Telescope data. Using a combined Bayesian Blocks and HOP algorithm, we identified seven high-quality orbital timescale flare segments from three sources and quantified 24 short-timescale flare structures. We then performed a comprehensive analysis of flare symmetry, power spectral density (PSD) of variability, and flux-photon index relation. The main results are as follows: (1) The flare symmetry parameter $A$ shows a "U-shaped" distribution. Short timescale flares are symmetric while long timescale flares are asymmetric. The number of fast-rise slow-decay and slow-rise fast-decay type flares are equal. No correlation is found between $A$ and peak/integral flux. No parameter evolution is seen between consecutive flares either. The observations support a scenario where longer timescale flares originate from superposition of short, symmetric sub-hour flares. (2) PSD from yearly to hourly timescales is modeled using the CARMA process. At lower frequencies, the PSD follows the typical broken power-law form. The high-frequency region of the PSD exhibits a continuous power-law shape, indicating that $\gamma$-ray variability originates from a single physical process across all probed timescales. (3) The flux-photon index distribution shows a pattern of "harder-when-brighter" or "softer-when-brighter," but becomes flat above a certain critical flux, with $\Gamma$ $\approx$ 2. This behavior cannot be simply explained by a two-component or blazar sequence model, and we speculate it may be related to complex interplay between electron acceleration and cooling., Comment: 14 pages, 5 figures, 2 tables, accepted for publication in ApJ

Transient quasi-periodic oscillations (QPOs) are extremely interesting observational phenomena. However, the precise physical mechanisms leading to their generation are still hotly debated. We performed a systematic search for transient QPO signals using Weighted Wavelet Z-transforms on the gamma-ray light curves of 134 bright blazars with peak flux exceeding $1\times10^{-6}$~ph~cm$^{-2}$~s$^{-1}$ as monitored by Fermi-LAT. Artificial light curves were generated from the power spectral density and probability distribution functions of the original light curves to assess the significance level of transient QPO. We discuss several physical mechanisms that produce transient QPOs, with the helical jet model providing the best explanation. This study identified four new transient QPO events. Interestingly, repetitive transient QPOs are observed in PKS 0537-441, and nested transient QPOs are detected in PKS 1424-41. Additionally, we find that transient QPOs tend to occur in the flare state of the blazar. Finally, we estimate the incidence of transient QPO events to be only about 3\%., Comment: 17 pages, 7 figures, 3 tables, 1 appendix, upper review, comments welcome

By means of the data sets of the Panoramic Survey Telescope and Rapid Response System (Pan-STARRS), we investigate the relationship between the variability amplitude and luminosity at 5100 \AA, black hole mass, Eddington ratio, $ R_{\rm Fe \, II}$ ( the ratio of the flux of Fe II line within 4435-4685 \AA ~to the broad proportion of $\rm H\beta$ line) as well as $ R_{5007}$ (the ratio of the flux [O III] line to the total $\rm H\beta$ line) of the broad line Seyfert 1 (BLS1) and narrow line Seyfert 1 (NLS1) galaxies sample in g,r,i,z and y bands, respectively. We also analyze the similarities and differences of the variability characteristics between the BLS1 galaxies and NLS1 galaxies. The results are listed as follows. (1). The cumulative probability distribution of the variability amplitude shows that NLS1 galaxies are lower than that in BLS1 galaxies. (2). We analyze the dependence of the variability amplitude with the luminosity at 5100 \AA, black hole mass, Eddington ratio, $ R_{\rm Fe \,II}$ and $ R_{5007}$, respectively. We find significantly negative correlations between the variability amplitude and Eddington ratio, insignificant correlations with the luminosity at 5100 \AA. The results also show significantly positive correlations with the black hole mass and $ R_{5007}$, significantly negative correlations with $ R_{\rm Fe\, II}$ which are consistent with Rakshit and Stalin(2017) in low redshift bins (z<0.4) and Ai et al.(2010). (3). The relationship between the variability amplitude and the radio loudness is investigated for 155 BLS1 galaxies and 188 NLS1 galaxies. No significant correlations are found in our results., Comment: 10 pages, 5 figures, accepted by Astrophysics and Space Science, in Press

Recent empirical evidence indicates that transformer based in-context learning performs better when using a prefix language model (prefixLM), in which in-context samples can all attend to each other, compared to causal language models (causalLM), which use auto-regressive attention that prohibits in-context samples to attend to future samples. While this result is intuitive, it is not understood from a theoretical perspective. In this paper we take a theoretical approach and analyze the convergence behavior of prefixLM and causalLM under a certain parameter construction. Our analysis shows that both LM types converge to their stationary points at a linear rate, but that while prefixLM converges to the optimal solution of linear regression, causalLM convergence dynamics follows that of an online gradient descent algorithm, which is not guaranteed to be optimal even as the number of samples grows infinitely. We supplement our theoretical claims with empirical experiments over synthetic and real tasks and using various types of transformers. Our experiments verify that causalLM consistently underperforms prefixLM in all settings., Comment: ICLR 2024 conference paper. Code available at: https://github.com/google-research/causallm_icl

We study the general physical properties of Fermi blazars using the Fermi fourth source catalog data (4FGL-DR2). The quasi-simultaneous multiwavelength data of Fermi blazar are fitted by using the one-zone leptonic model to obtain some physical parameters, such as jet power, magnetic field and Doppler factor. We study the distributions of the derived physical parameter as a function of black hole mass and accretion disk luminosity. The main results are as follows. (1) For a standard thin accretion disk, the jet kinetic power of most FSRQs can be explained by the BP mechanism. However, the jet kinetic power of most BL Lacs can not be explained by both the BZ mechanism or the BP mechanism. The BL Lacs may have ADAFs surrounding their massive black holes. (2) After excluding the redshift, there is a moderately strong correlation between the jet kinetic power and jet radiation power and the accretion disk luminosity for Fermi blazars. These results confirm a close connection between jet and accretion. The jet kinetic power is slightly larger than the accretion disk luminosity for Fermi blazars. (3) There is a significant correlation between jet kinetic power and gamma-ray luminosity and radio luminosity for Fermi blazars, which suggests that gamma-ray luminosity and radio luminosity can be used to indicate the jet kinetic power., Comment: 13pages,8 figures, accept for publication in ApJS

Disaggregated memory is a promising approach that addresses the limitations of traditional memory architectures by enabling memory to be decoupled from compute nodes and shared across a data center. Cloud platforms have deployed such systems to improve overall system memory utilization, but performance can vary across workloads. High-performance computing (HPC) is crucial in scientific and engineering applications, where HPC machines also face the issue of underutilized memory. As a result, improving system memory utilization while understanding workload performance is essential for HPC operators. Therefore, learning the potential of a disaggregated memory system before deployment is a critical step. This paper proposes a methodology for exploring the design space of a disaggregated memory system. It incorporates key metrics that affect performance on disaggregated memory systems: memory capacity, local and remote memory access ratio, injection bandwidth, and bisection bandwidth, providing an intuitive approach to guide machine configurations based on technology trends and workload characteristics. We apply our methodology to analyze thirteen diverse workloads, including AI training, data analysis, genomics, protein, fusion, atomic nuclei, and traditional HPC bookends. Our methodology demonstrates the ability to comprehend the potential and pitfalls of a disaggregated memory system and provides motivation for machine configurations. Our results show that eleven of our thirteen applications can leverage injection bandwidth disaggregated memory without affecting performance, while one pays a rack bisection bandwidth penalty and two pay the system-wide bisection bandwidth penalty. In addition, we also show that intra-rack memory disaggregation would meet the application's memory requirement and provide enough remote memory bandwidth., Comment: The submission builds on the following conference paper: N. Ding, S. Williams, H.A. Nam, et al. Methodology for Evaluating the Potential of Disaggregated Memory Systems,2nd International Workshop on RESource DISaggregation in High-Performance Computing (RESDIS), November 18, 2022. It is now submitted to the CCPE journal for review

We study the radio galaxies with known redshift detected by the Fermi satellite after 10 years of data (4FGL-DR2). We use a one-zone leptonic model to fit the quasi-simultaneous multiwavelength data of these radio galaxies and study the distributions of the derived physical parameter as a function of black hole mass and accretion disk luminosity. The main results are as follows. (1) We find that the jet kinetic power of most radio galaxies can be explained by the hybrid jet model based on ADAFs surrounding Kerr black holes. (2) After excluding the redshift, there is a significant correlation between the radiation jet power and the accretion disk luminosity, while the jet kinetic power is weakly correlated with the accretion disk luminosity. (3) We also find a significant correlation between inverse Compton luminosity and synchrotron luminosity. The slope of the correlation for radio galaxies is consistent with the synchrotron self-Compton (SSC) process. The result may suggest that the high-energy component of radio galaxies is dominated by the SSC process., Comment: 9 pages,7 figures, accept for publication in ApJS

We presented the multiwavelength analysis of a heavily obscured active galactic nucleus (AGN) in NGC 449. We first constructed a broadband X-ray spectrum using the latest NuSTAR and XMM-Newton data. Its column density ($\simeq 10^{24} \rm{cm}^{-2}$) and photon index ($\Gamma\simeq 2.4$) were reliably obtained by analyzing the broadband X-ray spectrum. However, the scattering fraction and the intrinsic X-ray luminosity could not be well constrained. Combined with the information obtained from the mid-infrared (mid-IR) spectrum and spectral energy distribution (SED) fitting, we derived its intrinsic X-ray luminosity ($\simeq 8.54\times 10^{42} \ \rm{erg\ s}^{-1}$) and scattering fraction ($f_{\rm{scat}}\simeq 0.26\%$). In addition, we also derived the following results: (1). The mass accretion rate of central AGN is about $2.54 \times 10^{-2} \rm{M}_\odot\ \rm{yr}^{-1}$, and the Eddington ratio is $8.39\times 10^{-2}$; (2). The torus of this AGN has a high gas-to-dust ratio ($N_{\rm H}/A_{\rm V}=8.40\times 10^{22}\ \rm{cm}^{-2}\ \rm{mag}^{-1}$); (3). The host galaxy and the central AGN are both in the early stage of co-evolution., Comment: 12 pages, 5 figures, 3 tables, Accepted to PASP

Recent research in robust optimization has shown an overfitting-like phenomenon in which models trained against adversarial attacks exhibit higher robustness on the training set compared to the test set. Although previous work provided theoretical explanations for this phenomenon using a robust PAC-Bayesian bound over the adversarial test error, related algorithmic derivations are at best only loosely connected to this bound, which implies that there is still a gap between their empirical success and our understanding of adversarial robustness theory. To close this gap, in this paper we consider a different form of the robust PAC-Bayesian bound and directly minimize it with respect to the model posterior. The derivation of the optimal solution connects PAC-Bayesian learning to the geometry of the robust loss surface through a Trace of Hessian (TrH) regularizer that measures the surface flatness. In practice, we restrict the TrH regularizer to the top layer only, which results in an analytical solution to the bound whose computational cost does not depend on the network depth. Finally, we evaluate our TrH regularization approach over CIFAR-10/100 and ImageNet using Vision Transformers (ViT) and compare against baseline adversarial robustness algorithms. Experimental results show that TrH regularization leads to improved ViT robustness that either matches or surpasses previous state-of-the-art approaches while at the same time requires less memory and computational cost.

Effective scaling and a flexible task interface enable large language models to excel at many tasks. We present PaLI (Pathways Language and Image model), a model that extends this approach to the joint modeling of language and vision. PaLI generates text based on visual and textual inputs, and with this interface performs many vision, language, and multimodal tasks, in many languages. To train PaLI, we make use of large pre-trained encoder-decoder language models and Vision Transformers (ViTs). This allows us to capitalize on their existing capabilities and leverage the substantial cost of training them. We find that joint scaling of the vision and language components is important. Since existing Transformers for language are much larger than their vision counterparts, we train a large, 4-billion parameter ViT (ViT-e) to quantify the benefits from even larger-capacity vision models. To train PaLI, we create a large multilingual mix of pretraining tasks, based on a new image-text training set containing 10B images and texts in over 100 languages. PaLI achieves state-of-the-art in multiple vision and language tasks (such as captioning, visual question-answering, scene-text understanding), while retaining a simple, modular, and scalable design., Comment: ICLR 2023 (Notable-top-5%)

Visual Question Answering (VQA) has benefited from increasingly sophisticated models, but has not enjoyed the same level of engagement in terms of data creation. In this paper, we propose a method that automatically derives VQA examples at volume, by leveraging the abundance of existing image-caption annotations combined with neural models for textual question generation. We show that the resulting data is of high-quality. VQA models trained on our data improve state-of-the-art zero-shot accuracy by double digits and achieve a level of robustness that lacks in the same model trained on human-annotated VQA data., Comment: 2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL 2022)

With the increasing abundance of pretrained models in recent years, the problem of selecting the best pretrained checkpoint for a particular downstream classification task has been gaining increased attention. Although several methods have recently been proposed to tackle the selection problem (e.g. LEEP, H-score), these methods resort to applying heuristics that are not well motivated by learning theory. In this paper we present PACTran, a theoretically grounded family of metrics for pretrained model selection and transferability measurement. We first show how to derive PACTran metrics from the optimal PAC-Bayesian bound under the transfer learning setting. We then empirically evaluate three metric instantiations of PACTran on a number of vision tasks (VTAB) as well as a language-and-vision (OKVQA) task. An analysis of the results shows PACTran is a more consistent and effective transferability measure compared to existing selection methods., Comment: European Conference on Computer Vision 2022 (oral)

Very little work has been done on star formation in dwarf lenticular galaxies (S0s). We present 2D-spectroscopic and millimetre observations made by Centro Astronomico Hispano Aleman (CAHA) 3.5 m optical and the IRAM-30 m millimetre telescopes, respectively, for a sample of four dwarf S0 galaxies with multiple star formation regions in the field environment. We find that although most of the sources deviate from the star forming main sequence relation, they all follow the Kennicutt-Schmidt law. After comparing the stellar and Halpha kinematics, we find that the velocity fields of both stars and ionized gas do not show regular motion and the velocity dispersions of stars and ionized gas are low in the regions with high star formation, suggesting these star-forming S0 galaxies still have significant rotation. This view can be supported by the result that most of these dwarf S0 galaxies are classified as fast rotators. The ratio of average atomic gas mass to stellar mass (~ 47%) is much greater than that of molecular gas mass to stellar mass (~ 1%). In addition, the gas-phase metallicities in the star-forming regions are lower than that of the non-star-forming regions. These results indicate that the extended star formation may originate from the combination of abundant atomic hydrogen, long dynamic time scale and low-density environment., Comment: 13 pages, 11 figures, Accepted by MNRAS

We present for the first time the timing and spectral analyses for a narrow-line Seyfert 1 galaxy, SBS 1353+564, using \it{XMM-Newton} and \it{Swift} multi-band observations from 2007 to 2019. Our main results are as follows: 1) The temporal variability of SBS 1353+564 is random, while the hardness ratio is relatively constant over a time span of 13 years; 2) We find a prominent soft X-ray excess feature below 2 keV, which cannot be well described by a simple blackbody component; 3) After comparing the two most prevailing models for interpreting the origin of the soft X-ray excess, we find that the relativistically smeared reflection model is unable to fit the data above 5 keV well and the X-ray spectra do not show any reflection features, such as the Fe K\alpha emission line. However, the warm corona model can obtain a good fitting result. For the warm corona model, we try to use three different sets of spin values to fit the data and derive different best-fitting parameter sets; 4) We compare the UV/optical spectral data with the extrapolated values of the warm corona model to determine which spin value is more appropriate for this source, and we find that the warm corona model with non-spin can sufficiently account for the soft X-ray excess in SBS 1353+564., Comment: 12 pages, 8 figures, accepted for publication in MNRAS

We have searched quasi-periodic oscillations (QPOs) for BL Lac PKS J2134-0153 in the 15 GHz radio light curve announced by the Owens Valley Radio Observatory 40-m telescope during the period from 2008-01-05 to 2019-05-18, utilizing the Lomb-Scargle periodogram (LSP) and the weighted wavelet Z-transform (WWZ) techniques. This is the first time that to search for periodic radio signal in BL Lac PKS J2134-0153 by these two methods. These two methods consistently reveal a QPO of 4.69 $\pm$ 0.14 years (>5 $\sigma$ confidence level). We discuss possible causes for this QPO, and we expected that the binary black holes scenario, where the QPO is caused by the precession of the binary black holes, is the most likely explanation. BL Lac PKS J2134-0153 thus could be a good binary black hole candidate. In the binary black holes scenario, the distance between the primary black hole and the secondary black hole is 1.83$\times$10$^{16}$ cm., Comment: 6 pages, 4 figures, Accepted for publication in MNRAS. arXiv admin note: text overlap with arXiv:2009.14604

Despite recent advances in its theoretical understanding, there still remains a significant gap in the ability of existing PAC-Bayesian theories on meta-learning to explain performance improvements in the few-shot learning setting, where the number of training examples in the target tasks is severely limited. This gap originates from an assumption in the existing theories which supposes that the number of training examples in the observed tasks and the number of training examples in the target tasks follow the same distribution, an assumption that rarely holds in practice. By relaxing this assumption, we develop two PAC-Bayesian bounds tailored for the few-shot learning setting and show that two existing meta-learning algorithms (MAML and Reptile) can be derived from our bounds, thereby bridging the gap between practice and PAC-Bayesian theories. Furthermore, we derive a new computationally-efficient PACMAML algorithm, and show it outperforms existing meta-learning algorithms on several few-shot benchmark datasets., Comment: Neural Information Processing Systems 2021

Theoretical models show that the power of relativistic jets of active galactic nuclei depends on the spin and mass of the central supermassive black holes, as well as the accretion. Here we report an analysis of archival observations of a sample of blazars. We find a significant correlation between jet kinetic power and the spin of supermassive black holes. At the same time, we use multiple linear regression to analyze the relationship between jet kinetic power and accretion, spin and black hole mass. We find that the spin of supermassive black holes and accretion are the most important contribution to the jet kinetic power. The contribution rates of both the spin of supermassive black holes and accretion are more than 95\%. These results suggest that the spin energy of supermassive black holes powers the relativistic jets. The jet production efficiency of almost all Fermi blazars can be explained by moderately thin magnetically arrested accretion disks around rapidly spinning black holes., Comment: 15pages, 8figures, accepted for publication in ApJ

The research community has proposed copious modifications to the Transformer architecture since it was introduced over three years ago, relatively few of which have seen widespread adoption. In this paper, we comprehensively evaluate many of these modifications in a shared experimental setting that covers most of the common uses of the Transformer in natural language processing. Surprisingly, we find that most modifications do not meaningfully improve performance. Furthermore, most of the Transformer variants we found beneficial were either developed in the same codebase that we used or are relatively minor changes. We conjecture that performance improvements may strongly depend on implementation details and correspondingly make some recommendations for improving the generality of experimental results., Comment: To appear at EMNLP 2021 as a conference paper

The availability of large-scale image captioning and visual question answering datasets has contributed significantly to recent successes in vision-and-language pre-training. However, these datasets are often collected with overrestrictive requirements inherited from their original target tasks (e.g., image caption generation), which limit the resulting dataset scale and diversity. We take a step further in pushing the limits of vision-and-language pre-training data by relaxing the data collection pipeline used in Conceptual Captions 3M (CC3M) [Sharma et al. 2018] and introduce the Conceptual 12M (CC12M), a dataset with 12 million image-text pairs specifically meant to be used for vision-and-language pre-training. We perform an analysis of this dataset and benchmark its effectiveness against CC3M on multiple downstream tasks with an emphasis on long-tail visual recognition. Our results clearly illustrate the benefit of scaling up pre-training data for vision-and-language tasks, as indicated by the new state-of-the-art results on both the nocaps and Conceptual Captions benchmarks., Comment: IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2021). Our dataset is available at https://github.com/google-research-datasets/conceptual-12m

We use the third catalog of blazars detected by Fermi/LAT (3LAC) and gamma-ray Narrow-line Seyfert 1 Galaxies (gamma-NLSy1s) to study the blazar sequence and relationship between them. Our results are as follows: (i) There is a weak anti-correlation between synchrotron peak frequency and peak luminosity for both Fermi blazars and gamma-NLSy1s, which supports the blazar sequence. However, after Doppler correction, the inverse correlation disappeared, which suggests that anti-correlation between synchrotron peak frequency and peak luminosity is affected by the beaming effect. (ii) There is a significant anti-correlation between jet kinetic power and synchrotron peak frequency for both Fermi blazars and gamma-NLSy1s, which suggests that the gamma-NLSy1s could fit well into the original blazar sequence. (iii) According to previous work, the relationship between synchrotron peak frequency and synchrotron curvature can be explained by statistical or stochastic acceleration mechanisms. There are significant correlations between synchrotron peak frequency and synchrotron curvature for whole sample, Fermi blazars and BL Lacs, respectively. The slopes of the correlation are consistent with statistical acceleration. For FSRQs, LBLs, IBLs, HBLs, and gamma-NLS1s, we also find a significant correlation, but in these cases the slopes can not be explained by previous theoretical models. (iv) The slope of relation between synchrotron peak frequency and synchrotron curvature in gamma-NLS1s is large than that of FSRQs and BL Lacs. This result may imply that the cooling dominates over the acceleration process for FSRQs and BL Lacs, while gamma-NLS1s is the opposite., Comment: 10 pages, 5 figures, 3 tables, Accepted for publication in ApJ

Even in deep X-ray surveys, Compton-thick active galactic nuclei (CT AGNs, ${\rm N_H} \geqslant 1.5~\times~10^{24}~{\rm cm}^{-2}$) are difficult to be identified due to X-ray flux suppression and their complex spectral shape. However, the study of CT AGNs is vital for understanding the rapid growth of black holes and the origin of cosmic X-ray background. In the local universe, the fraction of CT AGNs accounts for 30% of the whole AGN population. We may expect a higher fraction of CT AGNs in deep X-ray surveys, however, only 10% of AGNs have been identified as CT AGNs in the 7 Ms \textit{Chandra} Deep Field-South (CDFS) survey. In this work, we select 51 AGNs with abundant multi-wavelength data. Using the method of the mid-infrared (mid-IR) excess, we select hitherto unknown 8 CT AGN candidates in our sample. Seven of these candidates can confirm as CT AGN based on the multi-wavelength identification approach, and a new CT AGN (XID 133) is identified through the mid-IR diagnostics. We also discuss the X-ray origin of these eight CT AGNs and the reason why their column densities were underestimated in previous studies. We find that the multi-wavelength approaches of selecting CT AGNs are highly efficient, provided the high quality of observational data. We also find that CT AGNs have a higher Eddington ratio than non-CT AGNs, and that both CT AGNs and non-CT AGNs show similar properties of host galaxies., Comment: 13 pages, 6 figures, 3 tables, accepted for publication in ApJ

In this work, we investigate the 2014-2015 neutrino flare associated with the blazar TXS 0506+056 and a recently discovered muon neutrino event IceCube-200107A in spatial coincidence with the blazar 4FGL J0955.1+3551, under the framework of a two-zone radiation model of blazars where an inner/outer blob close to/far from the supermassive black hole are invoked. An interesting feature that the two sources share in common is that no evidence of GeV gamma-ray activity is found during the neutrino detection period, probably implying a large opacity for GeV gamma rays in the neutrino production region. In our model, continuous particle acceleration/injection takes place in the inner blob at the jet base, where the hot X-ray corona of the supermassive black hole provides target photon fields for efficient neutrino production and strong GeV gamma-ray absorption. We show that this model can self-consistently interpret the neutrino emission from both two blazars in a large parameter space. In the meantime, the dissipation processes in outer blob are responsible for the simultaneous multi-wavelength emission of both sources. In agreement with previous studies of TXS 0506+056 and, an intense MeV emission from the induced electromagnetic cascade in the inner blob is robustly expected to accompany the neutrino flare in our model could be used to test the model with the next-generation MeV gamma-ray detector in the future., Comment: 11 pages, 5 figures, accepted in ApJ

Recent advances in automatic evaluation metrics for text have shown that deep contextualized word representations, such as those generated by BERT encoders, are helpful for designing metrics that correlate well with human judgements. At the same time, it has been argued that contextualized word representations exhibit sub-optimal statistical properties for encoding the true similarity between words or sentences. In this paper, we present two techniques for improving encoding representations for similarity metrics: a batch-mean centering strategy that improves statistical properties; and a computationally efficient tempered Word Mover Distance, for better fusion of the information in the contextualized word representations. We conduct numerical experiments that demonstrate the robustness of our techniques, reporting results over various BERT-backbone learned metrics and achieving state of the art correlation with human ratings on several benchmarks., Comment: EMNLP 2020 Eval4NLP Workshop

Sequence generation models trained with teacher-forcing suffer from issues related to exposure bias and lack of differentiability across timesteps. Our proposed method, Teacher-Forcing with N-grams (TeaForN), addresses both these problems directly, through the use of a stack of N decoders trained to decode along a secondary time axis that allows model parameter updates based on N prediction steps. TeaForN can be used with a wide class of decoder architectures and requires minimal modifications from a standard teacher-forcing setup. Empirically, we show that TeaForN boosts generation quality on one Machine Translation benchmark, WMT 2014 English-French, and two News Summarization benchmarks, CNN/Dailymail and Gigaword., Comment: to be published in EMNLP 2020

In this work, we have searched quasi-periodic oscillations (QPOs) in the 15 GHz light curve of the FSRQ PKS J0805-0111 monitored by the Owens Valley Radio Observatory (OVRO) 40 m telescope during the period from January 9,2008 to May 9,2019, using the weighted wavelet Z-transform (WWZ) and the Lomb-Scargle Periodogram (LSP) techniques. This is the first time to search for periodic radio signal in the FSRQ PKS J0805-0111 by these two methods. All two methods consistently reveal a repeating signal with a periodicity of 3.38$\pm$ 0.8 years ($>$99.7\% confidence level). In order to determining the significance of the periods, the false alarm probability method was used, and a large number of Monte Carlo simulations were performed. As possible explanations, we discuss a number of scenarios including the thermal instability of thin disks scenario, the spiral jet scenario and the binary supermassive black hole scenario, we expected that the binary black hole scenario, where the QPO is caused by the precession of the binary black holes, is the most likely explanation. FSRQ PKS J0805-0111 thus could be a good binary black hole candidate. In the binary black hole scenario, the distance between the primary black hole and the secondary black hole is about $1.71\times 10^{16}$ cm.

Models based on the Transformer architecture have achieved better accuracy than the ones based on competing architectures for a large set of tasks. A unique feature of the Transformer is its universal application of a self-attention mechanism, which allows for free information flow at arbitrary distances. Following a probabilistic view of the attention via the Gaussian mixture model, we find empirical evidence that the Transformer attention tends to "explain away" certain input neurons. To compensate for this, we propose a doubly-normalized attention scheme that is simple to implement and provides theoretical guarantees for avoiding the "explaining away" effect without introducing significant computational or memory cost. Empirically, we show that the new attention schemes result in improved performance on several well-known benchmarks.

We introduce "talking-heads attention" - a variation on multi-head attention which includes linearprojections across the attention-heads dimension, immediately before and after the softmax operation.While inserting only a small number of additional parameters and a moderate amount of additionalcomputation, talking-heads attention leads to better perplexities on masked language modeling tasks, aswell as better quality when transfer-learning to language comprehension and question answering tasks.

We continuously monitored the blazar S5 0716+714 in the optical $g$, $r$ and $i$ bands from Nov. 10, 2017 to Jun. 06, 2019. The total number of observations is 201 nights including 26973 data points. This is a very large quasi-simultaneous multicolor sample for the blazar. The average time spans and time resolutions are 3.4 hours and 2.9 minutes per night, respectively. During the period of observations, the target source in the $r$ band brightens from $14^{\rm m}.16$ to $12^{\rm m}.29$ together with five prominent sub-flares, and then first becomes fainter to $14^{\rm m}.76$ and again brightens to $12^{\rm m}.94$ with seven prominent sub-flares. For the long-term variations, we find a strong flatter when brighter (FWB) trend at a low flux state and then a weak FWB trend at a higher flux state. A weak FWB trend at a low flux state and then a strong FWB trend at a higher flux state are also reported. Most of sub-flares show the strong FWB trends, except for two flares with a weak FWB trend. The particle acceleration and cooling mechanisms together with the superposition of different FWB-slopes from sub-flares are likely to explain the optical color behaviours. A scenario of bent jet is discussed., Comment: 27 pages, 9 figures, 4 tables, accepted for publication in ApJS

Pairwise sequence alignment is one of the most computationally intensive kernels in genomic data analysis, accounting for more than 90% of the runtime for key bioinformatics applications. This method is particularly expensive for third-generation sequences due to the high computational cost of analyzing sequences of length between 1Kb and 1Mb. Given the quadratic overhead of exact pairwise algorithms for long alignments, the community primarily relies on approximate algorithms that search only for high-quality alignments and stop early when one is not found. In this work, we present the first GPU optimization of the popular X-drop alignment algorithm, that we named LOGAN. Results show that our high-performance multi-GPU implementation achieves up to 181.6 GCUPS and speed-ups up to 6.6x and 30.7x using 1 and 6 NVIDIA Tesla V100, respectively, over the state-of-the-art software running on two IBM Power9 processors using 168 CPU threads, with equivalent accuracy. We also demonstrate a 2.3x LOGAN speed-up versus ksw2, a state-of-art vectorized algorithm for sequence alignment implemented in minimap2, a long-read mapping software. To highlight the impact of our work on a real-world application, we couple LOGAN with a many-to-many long-read alignment software called BELLA, and demonstrate that our implementation improves the overall BELLA runtime by up to 10.6x. Finally, we adapt the Roofline model for LOGAN and demonstrate that our implementation is near-optimal on the NVIDIA Tesla V100s.

In this report, the method for the iqiyi submission to the task of ActivityNet 2019 Kinetics-700 challenge is described. Three models are involved in the model ensemble stage: TSN, HG-NL and StNet. We propose the hierarchical group-wise non-local (HG-NL) module for frame-level features aggregation for video classification. The standard non-local (NL) module is effective in aggregating frame-level features on the task of video classification but presents low parameters efficiency and high computational cost. The HG-NL method involves a hierarchical group-wise structure and generates multiple attention maps to enhance performance. Basing on this hierarchical group-wise structure, the proposed method has competitive accuracy, fewer parameters and smaller computational cost than the standard NL. For the task of ActivityNet 2019 Kinetics-700 challenge, after model ensemble, we finally obtain an averaged top-1 and top-5 error percentage 28.444% on the test set., Comment: Tech report

The physical parameters of galaxies and/or AGNs can be derived by fitting their multi-band spectral energy distributions (SEDs). By using CIGALE code, we perform multi-band SED fitting (from ultraviolet to infrared) for 791 X-ray sources (518 AGNs and 273 normal galaxies) in the 7 Ms Chandra Deep Field-south survey (CDFS). We consider the contributions from AGNs and adopt more accurate redshifts than published before. Therefore, more accurate star formation rates (SFRs) and stellar masses (M$_*$) are derived. We classify the 518 AGNs into type-I and type-II based on their optical spectra and their SEDs. Moreover, six AGN candidates are selected from the 273 normal galaxies based on their SEDs. Our main results are as follows: (1) the host galaxies of AGNs have larger M$_*$ than normal galaxies, implying that AGNs prefer to host in massive galaxies; (2) the specific star formation rates (sSFRs) of AGN host galaxies are different from those of normal galaxies, suggesting that AGN feedback may play an important role in the star formation activity; (3) we find that the fraction of optically obscured AGNs in CDFS decreases with the increase of intrinsic X-ray luminosity, which is consistent with previous studies;(4) the host galaxies of type-I AGNs tend to have lower M$_*$ than type-II AGNs, which may suggest that dust in the host galaxy may also contribute to the optical obscuration of AGNs., Comment: Comments: 17 Pages, 16 Figures, 5 tables. Accepted for publication in MNRAS

We have selected a sample of nearby galaxies from Sloan Digital Sky Survey Data Release 7 (SDSS DR7) to investigate the physical properties variation from blue cloud to green valley to red sequence. The sample is limited in a narrow range in color-stellar mass diagram. After splitting green valley galaxies into two parts---a bluer green valley (green 1) and a redder one (green 2) and three stellar mass bins, we investigate the physical properties variation across the green valley region. Our main results are as following: (i) The percentages of pure bulge and bulge-dominated/elliptical galaxies increase gradually from blue cloud to red sequence while the percentages of pure disk and disk-dominated/spiral galaxies decrease gradually in all stellar mass bins and different environments; (ii) With the analysis of morphological and structural parameters (e.g., concentration (C) and the stellar mass surface density within the central 1Kpc ($\Sigma_{1}$)), red galaxies show the most luminous and compact cores than both green valley and blue galaxies while blue galaxies show the opposite behavior in all stellar mass bins. (iii) A strong negative (positive) relationship between bulge-to-total light ratio (B/T) and specific star formation rate (sSFR) ($D_{4000}$) is found from blue to red galaxies. Our results indicate that the growth of bulge plays an important role when the galaxies change from the blue cloud, to green valley, and to the red sequence.

Supervised training of abstractive language generation models results in learning conditional probabilities over language sequences based on the supervised training signal. When the training signal contains a variety of writing styles, such models may end up learning an 'average' style that is directly influenced by the training data make-up and cannot be controlled by the needs of an application. We describe a family of model architectures capable of capturing both generic language characteristics via shared model parameters, as well as particular style characteristics via private model parameters. Such models are able to generate language according to a specific learned style, while still taking advantage of their power to model generic language phenomena. Furthermore, we describe an extension that uses a mixture of output distributions from all learned styles to perform on-the fly style adaptation based on the textual input alone. Experimentally, we find that the proposed models consistently outperform models that encapsulate single-style or average-style language generation capabilities., Comment: NAACL 2018

Policy-gradient approaches to reinforcement learning have two common and undesirable overhead procedures, namely warm-start training and sample variance reduction. In this paper, we describe a reinforcement learning method based on a softmax value function that requires neither of these procedures. Our method combines the advantages of policy-gradient methods with the efficiency and simplicity of maximum-likelihood approaches. We apply this new cold-start reinforcement learning method in training sequence generation models for structured output prediction problems. Empirical evidence validates this method on automatic summarization and image captioning tasks., Comment: Conference on Neural Information Processing Systems 2017. Main paper and supplementary material

We introduce a new multi-modal task for computer systems, posed as a combined vision-language comprehension challenge: identifying the most suitable text describing a scene, given several similar options. Accomplishing the task entails demonstrating comprehension beyond just recognizing "keywords" (or key-phrases) and their corresponding visual concepts. Instead, it requires an alignment between the representations of the two modalities that achieves a visually-grounded "understanding" of various linguistic elements and their dependencies. This new task also admits an easy-to-compute and well-studied metric: the accuracy in detecting the true target among the decoys. The paper makes several contributions: an effective and extensible mechanism for generating decoys from (human-created) image captions; an instance of applying this mechanism, yielding a large-scale machine comprehension dataset (based on the COCO images and captions) that we make publicly available; human evaluation results on this dataset, informing a performance upper-bound; and several baseline and competitive learning approaches that illustrate the utility of the proposed task and dataset in advancing both image and language comprehension. We also show that, in a multi-task learning setting, the performance on the proposed task is positively correlated with the end-to-end task of image captioning., Comment: 11 pages

We present a family of neural-network--inspired models for computing continuous word representations, specifically designed to exploit both monolingual and multilingual text. This framework allows us to perform unsupervised training of embeddings that exhibit higher accuracy on syntactic and semantic compositionality, as well as multilingual semantic similarity, compared to previous models trained in an unsupervised fashion. We also show that such multilingual embeddings, optimized for semantic similarity, can improve the performance of statistical machine translation with respect to how it handles words not present in the parallel data., Comment: 12 pages

We present a dual contribution to the task of machine reading-comprehension: a technique for creating large-sized machine-comprehension (MC) datasets using paragraph-vector models; and a novel, hybrid neural-network architecture that combines the representation power of recurrent neural networks with the discriminative power of fully-connected multi-layered networks. We use the MC-dataset generation technique to build a dataset of around 2 million examples, for which we empirically determine the high-ceiling of human performance (around 91% accuracy), as well as the performance of a variety of computer models. Among all the models we have experimented with, our hybrid neural-network architecture achieves the highest performance (83.2% accuracy). The remaining gap to the human-performance ceiling provides enough room for future model improvements., Comment: 10 pages

Recent advances in Bayesian learning with large-scale data have witnessed emergence of stochastic gradient MCMC algorithms (SG-MCMC), such as stochastic gradient Langevin dynamics (SGLD), stochastic gradient Hamiltonian MCMC (SGHMC), and the stochastic gradient thermostat. While finite-time convergence properties of the SGLD with a 1st-order Euler integrator have recently been studied, corresponding theory for general SG-MCMCs has not been explored. In this paper we consider general SG-MCMCs with high-order integrators, and develop theory to analyze finite-time convergence properties and their asymptotic invariant measures. Our theoretical results show faster convergence rates and more accurate invariant measures for SG-MCMCs with higher-order integrators. For example, with the proposed efficient 2nd-order symmetric splitting integrator, the {\em mean square error} (MSE) of the posterior average for the SGHMC achieves an optimal convergence rate of $L^{-4/5}$ at $L$ iterations, compared to $L^{-2/3}$ for the SGHMC and SGLD with 1st-order Euler integrators. Furthermore, convergence results of decreasing-step-size SG-MCMCs are also developed, with the same convergence rates as their fixed-step-size counterparts for a specific decreasing sequence. Experiments on both synthetic and real datasets verify our theory, and show advantages of the proposed method in two large-scale real applications., Comment: NIPS 2015

Stochastic gradient MCMC (SG-MCMC) has played an important role in large-scale Bayesian learning, with well-developed theoretical convergence properties. In such applications of SG-MCMC, it is becoming increasingly popular to employ distributed systems, where stochastic gradients are computed based on some outdated parameters, yielding what are termed stale gradients. While stale gradients could be directly used in SG-MCMC, their impact on convergence properties has not been well studied. In this paper we develop theory to show that while the bias and MSE of an SG-MCMC algorithm depend on the staleness of stochastic gradients, its estimation variance (relative to the expected estimate, based on a prescribed number of samples) is independent of it. In a simple Bayesian distributed system with SG-MCMC, where stale gradients are computed asynchronously by a set of workers, our theory indicates a linear speedup on the decrease of estimation variance w.r.t. the number of workers. Experiments on synthetic data and deep neural networks validate our theory, demonstrating the effectiveness and scalability of SG-MCMC with stale gradients., Comment: NIPS2016

A critical question for the field of quantum computing in the near future is whether quantum devices without error correction can perform a well-defined computational task beyond the capabilities of state-of-the-art classical computers, achieving so-called quantum supremacy. We study the task of sampling from the output distributions of (pseudo-)random quantum circuits, a natural task for benchmarking quantum computers. Crucially, sampling this distribution classically requires a direct numerical simulation of the circuit, with computational cost exponential in the number of qubits. This requirement is typical of chaotic systems. We extend previous results in computational complexity to argue more formally that this sampling task must take exponential time in a classical computer. We study the convergence to the chaotic regime using extensive supercomputer simulations, modeling circuits with up to 42 qubits - the largest quantum circuits simulated to date for a computational task that approaches quantum supremacy. We argue that while chaotic states are extremely sensitive to errors, quantum supremacy can be achieved in the near-term with approximately fifty superconducting qubits. We introduce cross entropy as a useful benchmark of quantum circuits which approximates the circuit fidelity. We show that the cross entropy can be efficiently measured when circuit simulations are available. Beyond the classically tractable regime, the cross entropy can be extrapolated and compared with theoretical estimates of circuit fidelity to define a practical quantum supremacy test., Comment: Increased circuit depth, added one author, updated references. 23 pages, 15 figures

Quantum annealing (QA) has been proposed as a quantum enhanced optimization heuristic exploiting tunneling. Here, we demonstrate how finite range tunneling can provide considerable computational advantage. For a crafted problem designed to have tall and narrow energy barriers separating local minima, the D-Wave 2X quantum annealer achieves significant runtime advantages relative to Simulated Annealing (SA). For instances with 945 variables, this results in a time-to-99%-success-probability that is $\sim 10^8$ times faster than SA running on a single processor core. We also compared physical QA with Quantum Monte Carlo (QMC), an algorithm that emulates quantum tunneling on classical processors. We observe a substantial constant overhead against physical QA: D-Wave 2X again runs up to $\sim 10^8$ times faster than an optimized implementation of QMC on a single core. We note that there exist heuristic classical algorithms that can solve most instances of Chimera structured problems in a timescale comparable to the D-Wave 2X. However, we believe that such solvers will become ineffective for the next generation of annealers currently being designed. To investigate whether finite range tunneling will also confer an advantage for problems of practical interest, we conduct numerical studies on binary optimization problems that cannot yet be represented on quantum hardware. For random instances of the number partitioning problem, we find numerically that QMC, as well as other algorithms designed to simulate QA, scale better than SA. We discuss the implications of these findings for the design of next generation quantum annealers., Comment: 17 pages, 13 figures. Edited for clarity, in part in response to comments. Added link to benchmark instances

We propose a totally corrective boosting algorithm with explicit cardinality regularization. The resulting combinatorial optimization problems are not known to be efficiently solvable with existing classical methods, but emerging quantum optimization technology gives hope for achieving sparser models in practice. In order to demonstrate the utility of our algorithm, we use a distributed classical heuristic optimizer as a stand-in for quantum hardware. Even though this evaluation methodology incurs large time and resource costs on classical computing machinery, it allows us to gauge the potential gains in generalization performance and sparsity of the resulting boosted ensembles. Our experimental results on public data sets commonly used for benchmarking of boosting algorithms decidedly demonstrate the existence of such advantages. If actual quantum optimization were to be used with this algorithm in the future, we would expect equivalent or superior results at much smaller time and energy costs during training. Moreover, studying cardinality-penalized boosting also sheds light on why unregularized boosting algorithms with early stopping often yield better results than their counterparts with explicit convex regularization: Early stopping performs suboptimal cardinality regularization. The results that we present here indicate it is beneficial to explicitly solve the combinatorial problem still left open at early termination.

We consider classification problems in which the label space has structure. A common example is hierarchical label spaces, corresponding to the case where one label subsumes another (e.g., animal subsumes dog). But labels can also be mutually exclusive (e.g., dog vs cat) or unrelated (e.g., furry, carnivore). To jointly model hierarchy and exclusion relations, the notion of a HEX (hierarchy and exclusion) graph was introduced in [7]. This combined a conditional random field (CRF) with a deep neural network (DNN), resulting in state of the art results when applied to visual object classification problems where the training labels were drawn from different levels of the ImageNet hierarchy (e.g., an image might be labeled with the basic level category "dog", rather than the more specific label "husky"). In this paper, we extend the HEX model to allow for soft or probabilistic relations between labels, which is useful when there is uncertainty about the relationship between two labels (e.g., an antelope is "sort of" furry, but not to the same degree as a grizzly bear). We call our new model pHEX, for probabilistic HEX. We show that the pHEX graph can be converted to an Ising model, which allows us to use existing off-the-shelf inference methods (in contrast to the HEX method, which needed specialized inference algorithms). Experimental results show significant improvements in a number of large-scale visual object classification tasks, outperforming the previous HEX model., Comment: International Conference on Computer Vision (2015)

Quantum annealing is a heuristic quantum algorithm which exploits quantum resources to minimize an objective function embedded as the energy levels of a programmable physical system. To take advantage of a potential quantum advantage, one needs to be able to map the problem of interest to the native hardware with reasonably low overhead. Because experimental considerations constrain our objective function to take the form of a low degree PUBO (polynomial unconstrained binary optimization), we employ non-convex loss functions which are polynomial functions of the margin. We show that these loss functions are robust to label noise and provide a clear advantage over convex methods. These loss functions may also be useful for classical approaches as they compile to regularized risk expressions which can be evaluated in constant time with respect to the number of training examples., Comment: 15 pages, 6 figures

Cancer cells maintain a high level of aerobic glycolysis (the Warburg effect), which is associated with their rapid proliferation. Many studies have reported that the suppression of glycolysis and activation of oxidative phosphorylation can repress the growth of cancer cells through regulation of key regulators. Whether Warburg effect of cancer cells could be switched by some other environmental stimulus? Herein, we report an interesting phenomenon in which cells alternated between glycolysis and mitochondrial respiration depending on the type of radiation they were exposed to. We observed enhanced glycolysis and mitochondrial respiration in HeLa cells exposed to 2-Gy X-ray and 2-Gy carbon ion radiation, respectively. This discovery may provide novel insights for tumor therapy.