Search

In many Reinforcement Learning (RL) papers, learning curves are useful indicators to measure the effectiveness of RL algorithms. However, the complete raw data of the learning curves are rarely available. As a result, it is usually necessary to reproduce the experiments from scratch, which can be time-consuming and error-prone. We present Open RL Benchmark, a set of fully tracked RL experiments, including not only the usual data such as episodic return, but also all algorithm-specific and system metrics. Open RL Benchmark is community-driven: anyone can download, use, and contribute to the data. At the time of writing, more than 25,000 runs have been tracked, for a cumulative duration of more than 8 years. Open RL Benchmark covers a wide range of RL libraries and reference implementations. Special care is taken to ensure that each experiment is precisely reproducible by providing not only the full parameters, but also the versions of the dependencies used to generate it. In addition, Open RL Benchmark comes with a command-line interface (CLI) for easy fetching and generating figures to present the results. In this document, we include two case studies to demonstrate the usefulness of Open RL Benchmark in practice. To the best of our knowledge, Open RL Benchmark is the first RL benchmark of its kind, and the authors hope that it will improve and facilitate the work of researchers in the field., Comment: Under review

The horizon of a classical black hole (BH), functioning as a one-way membrane, plays a vital role in the dynamic evolution of binary BHs, capable of absorbing fluxes entirely. Tidal heating, stemming from this phenomenon, exerts a notable influence on the production of gravitational waves(GWs). If at least one member of a binary is an exotic compact object (ECO) instead of a BH, the absorption of fluxes is expected to be incomplete and the tidal heating would be different. Thus, tidal heating can be utilized for model-independent investigations into the nature of compact object. In this paper, assuming that the extreme-mass-ratio inspiral (EMRI) contains a stellar-mass compact object orbiting around a massive ECO with a reflective surface, we compute the GWs from the generic EMRI orbits. Using the accurate and analytic flux formulas in the black hole spacetime, we adapted these formulas in the vicinity of the ECO surface by incorporating a reflectivity parameter. Under the adiabatic approximation, we can evolve the orbital parameters and compute the EMRI waveforms. The effect of tidal heating for the spinning and non-spinning objects can be used to constrain the reflectivity of the surface at the level of 10^-4 by computing the mismatch and fisher information matrix., Comment: 10 pages, 5 figures, 1 table

Space-based gravitational wave detectors like TianQin or LISA could observe extreme-mass-ratio-inspirals (EMRIs) at millihertz frequencies. The accurate identification of these EMRI signals from the data plays a crucial role in enabling in-depth study of astronomy and physics. We aim at the identification stage of the data analysis, with the aim to extract key features of the signal from the data, such as the evolution of the orbital frequency, as well as to pinpoint the parameter range that can fit the data well for the subsequent parameter inference stage. In this manuscript, we demonstrated the identification of EMRI signals without any additional prior information on physical parameters. High-precision measurements of EMRI signals have been achieved, using a hierarchical search. It combines the search for physical parameters that guide the subsequent parameter inference, and a semi-coherent search with phenomenological waveforms that reaches precision levels down to $10^{-4}$ for the phenomenological waveform parameters $\omega_{0}$, $\dot{\omega}_{0}$, and $\ddot{\omega}_{0}$. As a result, we obtain measurement relative errors of less than 4% for the mass of the massive black hole, while keeping the relative errors of the other parameters within as small as 0.5%., Comment: 15 pages, 6 figures, comments welcom

Space-based gravitational wave detectors such as TianQin, LISA, and TaiJi have the potential to outperform themselves through joint observation. To achieve this, it is desirable to practice joint data analysis in advance on simulated data that encodes the intrinsic correlation among the signals found in different detectors that operate simultaneously. In this paper, we introduce \texttt{GWSpace}, a package that can simulate the joint detection data from TianQin, LISA, and TaiJi. The software is not a groundbreaking work that starts from scratch. Rather, we use as many open-source resources as possible, tailoring them to the needs of simulating the multi-mission science data and putting everything into a ready-to-go and easy-to-use package. We shall describe the main components, the construction, and a few examples of application of the package. A common coordinate system, namely the Solar System Barycenter (SSB) coordinate system, is utilized to calculate spacecraft orbits for all three missions. The paper also provides a brief derivation of the detection process and outlines the general waveform of sources detectable by these detectors., Comment: 24 pages, 13 figures, GWSpace will be uploaded at https://github.com/TianQinSYSU/GWSpace

TianQin and LISA are space-based laser interferometer gravitational wave (GW) detectors planned to be launched in the mid-2030s. Both detectors will detect low-frequency GWs around $10^{-2}\,{\rm Hz}$, however, TianQin is more sensitive to frequencies above this common sweet-spot while LISA is more sensitive to frequencies below $10^{-2}\,{\rm Hz}$. Therefore, TianQin and LISA will be able to detect the same sources but with different accuracy depending on the source and its parameters. We consider some of the most important astrophysical sources -- massive black hole binaries, stellar-mass black hole binaries, double white dwarfs, extreme mass ratio inspirals, light and heavy intermediate mass ratio inspirals, as well as the stochastic gravitational background of astrophysical origin -- that TianQin and LISA will be able to detect. For each of these sources, we analyze how far they can be detected (detection distance) and how well their parameters can be measured (detection accuracy) using a Fisher Matrix analysis. We compare the results obtained by the three detection scenarios (TianQin alone, LISA alone, and joint detection by LISA and TianQin) highlighting the gains from joint detection as well as the contribution of TianQin and LISA to a combined study of astrophysical sources. In particular, we consider the different orientations, lifetimes, and duty cycles of the two detectors to explore how they can give a more complete picture when working together., Comment: 49 pages, 63 figures

Recently discovered regular X-ray bursts known as quasi-periodic eruptions have a proposed model that suggests a tidal stripping white dwarf inspiralling into the galaxy's central black hole on an eccentric orbit. According to this model, the interaction of the stripping white dwarf with the central black hole would emit gravitational wave signals as well, their detection can help explore the formation mechanism of quasi-periodic eruptions and facilitate multi-messenger observations. In this paper, we aim to perform a preliminary study of the gravitation wave observation of TianQin on this stripping white dwarf model. We investigated the horizon distance of TianQin on this type of gravitation wave signal and found it can be set to 200Mpc. We also find that those stripping white dwarf model sources with central black hole mass within $10^4\sim10^{5.5}M_\odot$ are more likely to be detected by TianQin. We assessed the parameter estimation precision of TianQin on those stripping white dwarf model sources. Our result shows that, even in the worst case, TianQin can determine the central black hole mass, the white dwarf mass, the central black hole spin, and the orbital initial eccentricity with a precision of $10^{-2}$. In the optimistic case, TianQin can determine the central black hole mass and the white dwarf mass with a precision of $10^{-7}$, determine the central black hole spin with a precision of $10^{-5}$, and determine the orbital initial eccentricity with a precision of $10^{-8}$. Moreover, TianQin can determine the luminosity distance with a precision of $10^{-1}$ and determine the sky localization with a precision of $10^{-2}\sim10$ $\rm deg^2$., Comment: 9 pages, 4 figures

This paper aims to study the local derivations, 2-local automorphisms and local automorphisms on the super-Virasoro algebras. The primary focus is to establish that every local derivation of the super-Virasoro algebras is indeed a derivation, and to demonstrate that every local or 2-local automorphism of the super-Virasoro algebras is an automorphism., Comment: Commun Alg. 2024

CleanRL is an open-source library that provides high-quality single-file implementations of Deep Reinforcement Learning algorithms. It provides a simpler yet scalable developing experience by having a straightforward codebase and integrating production tools to help interact and scale experiments. In CleanRL, we put all details of an algorithm into a single file, making these performance-relevant details easier to recognize. Additionally, an experiment tracking feature is available to help log metrics, hyperparameters, videos of an agent's gameplay, dependencies, and more to the cloud. Despite succinct implementations, we have also designed tools to help scale, at one point orchestrating experiments on more than 2000 machines simultaneously via Docker and cloud providers. Finally, we have ensured the quality of the implementations by benchmarking against a variety of environments. The source code of CleanRL can be found at https://github.com/vwxyzjn/cleanrl

The properties of the young pulsars and their relations to the supernova remnants (SNRs) have been the interesting topics. At present, 383 SNRs in the Milky Way galaxy have been published, which are associated with 64 radio pulsars and 46 pulsars with high energy emissions. However, we noticed that 630 young radio pulsars with spin periods of less than half a second have been not yet observed the SNRs surrounding or nearby them, which arises a question of that could the two types of young radio pulsars with/without SNRs hold distinctive characteristics? Here, we employ the statistical tests on the two groups of young radio pulsars with (52) and without (630) SNRs to reveal if they share different origins. Kolmogorov-Smirnov (K-S) and Mann-Whitney-Wilcoxon (M-W-W) tests indicate that the two samples have the different distributions with parameters of spin period ($P$), derivative of spin period ($\dot P$), surface magnetic field strength ($B$), and energy loss rate ($\dot E$). Meanwhile, the cumulative number ratio between the pulsars with and without SNRs at the different spindown ages decreases significantly after $\rm10-20\,Kyr$. So we propose that the existence of the two types of supernovae (SNe), corresponding to their SNR lifetimes, which can be roughly ascribed to the low-energy and high-energy SNe. Furthermore, the low-energy SNe may be formed from the $\rm8-12\,M_{\odot}$ progenitor, e.g., possibly experiencing the electron capture, while the main sequence stars of $\rm12-25\,M_{\odot}$ may produce the high-energy SNe probably by the iron core collapse., Comment: 8 pages, 10 figures, 3 tables. Published in MNRAS

Fast radio bursts (FRBs) are extremely strong radio flares lasting several milliseconds, most of which come from unidentified objects at a cosmological distance. They can be apparently repeating or not. In this paper, we analyzed 18 repeaters and 12 non-repeating FRBs observed in the frequency bands of 400-800 MHz from CHIME. We investigated the distributions of FRB isotropic-equivalent radio luminosity, considering the K correction. Statistically, the luminosity distribution can be better fitted by Gaussian form than by power-law. Based on the above results, together with the observed FRB event rate, pulse duration, and radio luminosity, FRB origin models are evaluated and constrained such that the gamma-ray bursts (GRBs) may be excluded for the non-repeaters while magnetars or neutron stars (NSs) emitting the supergiant pulses are preferred for the repeaters. We also found the necessity of a small FRB emission beaming solid angle (about 0.1 sr) from magnetars that should be considered, and/or the FRB association with soft gamma-ray repeaters (SGRs) may lie at a low probability of about 10%. Finally, we discussed the uncertainty of FRB luminosity caused by the estimation of the distance that is inferred by the simple relation between the redshift and dispersion measure (DM)., Comment: 12 pages, 2 figures, 2 tables. Accepted in Research in Astronomy and Astrophysics (RAA)

Fast Radio Bursts (FRBs) are the short, strong radio pulses lasting several milliseconds. They are subsequently identified, for the most part, as emanating from unknown objects at cosmological distances. At present, over one hundred FRBs have been verified, classified into two groups: repeating bursts (20 samples) and apparently non-repeating bursts (91 samples). Their origins, however, are still hotly debated. Here,we investigate the statistical classifications for the two groups of samples to see if the non-repeating and repeating FRBs have different origins by employing Anderson-Darling (A-D) test and Mann-Whitney-Wilcoxon (M-W-W) test. Firstly, by taking the pulse width as a statistical variant, we found that the repeating samples do not follow the Gaussian statistics (may belong to a chi-square distribution), although the overall data and non-repeating group do follow the Gaussian. Meanwhile, to investigate the statistical differences between the two groups, we turn to M-W-W test and notice that the two distributions have different origins. Secondly, we consider the FRB radio luminosity as a statistical variant, and find that both groups of samples can be regarded as the Gaussian distributions under the A-D test, although they have different origins according to M-W-W tests. Therefore, statistically, we can conclude that our classifications of both repeaters and non-repeaters are plausible, that the two FRB classes have different origins, or each has experienced distinctive phases or been subject to its own physical processes., Comment: 7 pages, 5 figures, 3 tables. Accepted for publication in MNRAS

Deep Reinforcement Learning (DRL) has shown impressive performance on domains with visual inputs, in particular various games. However, the agent is usually trained on a fixed environment, e.g. a fixed number of levels. A growing mass of evidence suggests that these trained models fail to generalize to even slight variations of the environments they were trained on. This paper advances the hypothesis that the lack of generalization is partly due to the input representation, and explores how rotation, cropping and translation could increase generality. We show that a cropped, translated and rotated observation can get better generalization on unseen levels of two-dimensional arcade games from the GVGAI framework. The generality of the agents is evaluated on both human-designed and procedurally generated levels., Comment: IEEE Conference on Games 2020 Full Paper

The evolution of neutron star (NS) magnetic field (B-field) has long been an important topic, which is still not yet settled down. Here, we analyze the NS B-fields inferred by the cyclotron resonance scattering features (CRSFs) for the high mass X-ray binaries (HMXBs) and by the magnetic dipole model for the spin-down pulsars. We find that the B-fields of both the 32 NSHMXBs and 28 young pulsars with the supernova remnants follow the log-normal distributions, with the average values of 3.4 * 10^12 G and 4.1 * 10^12 G respectively, which are further verified to come from the same continuous distribution by the statistical tests. These results declaim that the two methods of measuring NS B-fields are reliable for the above two groups of samples. In addition, since the NS-HMXBs have experienced the spin-down phase as the normal pulsars without accretion and then the spin-up phase by accretion, their ages should be about million years (Myrs). Our statistical facts imply that the B-fields of NS-HMXBs have little decayed in their non-accretion spin-down phases of ~ Myrs, as well as in their accretion phases of ~ 0.1Myrs., Comment: 8 pages, 2 figures, 4 tables, accepted by Astrophysics and Space Science