Search

Molecules have a number of distinct properties whose importance and application vary. Often, in reality, labels for some properties are hard to achieve despite their practical importance. A common solution to such data scarcity is to use models of good generalization with transfer learning. This involves domain experts for designing source and target tasks whose features are shared. However, this approach has limitations: i). Difficulty in accurate design of source-target task pairs due to the large number of tasks, and ii). corresponding computational burden verifying many trials and errors of transfer learning design, thereby iii). constraining the potential of foundation modeling of multi-task molecular property prediction. We address the limitations of the manual design of transfer learning via data-driven bi-level optimization. The proposed method enables scalable multi-task transfer learning for molecular property prediction by automatically obtaining the optimal transfer ratios. Empirically, the proposed method improved the prediction performance of 40 molecular properties and accelerated training convergence.

Training deep learning models on limited data while maintaining generalization is one of the fundamental challenges in molecular property prediction. One effective solution is transferring knowledge extracted from abundant datasets to those with scarce data. Recently, a novel algorithm called Geometrically Aligned Transfer Encoder (GATE) has been introduced, which uses soft parameter sharing by aligning the geometrical shapes of task-specific latent spaces. However, GATE faces limitations in scaling to multiple tasks due to computational costs. In this study, we propose a task addition approach for GATE to improve performance on target tasks with limited data while minimizing computational complexity. It is achieved through supervised multi-task pre-training on a large dataset, followed by the addition and training of task-specific modules for each target task. Our experiments demonstrate the superior performance of the task addition strategy for GATE over conventional multi-task methods, with comparable computational costs., Comment: 11 pages, 5 figures, Accepted at AI for Science Workshop at 41st International Conference on Machine Learning

Extending geodetic and astrometric Very Long Baseline Interferometry (VLBI) observations from traditional centimeter wavebands to millimeter wavebands offers numerous scientific potentials and benefits. However, it was considered quite challenging due to various factors, including the increased effects of atmospheric opacity and turbulence at millimeter wavelengths. Here, we present the results of the first geodetic-mode VLBI experiment, simultaneously observing 82 sources at 22/43/88/132 GHz (K/Q/W/D bands) using the Korean VLBI Network (KVN). We introduced the frequency phase transfer (FPT) method to geodetic VLBI analysis, an approach for calibrating atmospheric phase fluctuations at higher frequencies by transferring phase solutions from lower frequencies. With a 2-minute scan, FPT improved the signal-to-noise ratio (SNR) of most fringes, some by over 100%, thereby enhancing the detection rate of weak sources at millimeter wavebands. Additionally, FPT reduced systematic errors in group delay and delay rate, with the weighted root-mean-squares (WRMS) of the post-fitting residuals decreasing from 25.0 ps to 20.5 ps at the W band and from 39.3 ps to 27.6 ps at the D band. There were no notable differences observed in calibrating atmospheric phase fluctuations at the K band (WRMS = 12.4 ps) and Q band (WRMS = 11.8 ps) with the KVN baselines. This experiment demonstrated that the millimeter waveband can be used for geodetic and astrometric applications with high precision., Comment: 17 pages, 6 figures, accepted by The Astronomical Journal. Your feedback to improve the manuscript would be greatly appreciated

Molecular datasets often suffer from a lack of data. It is well-known that gathering data is difficult due to the complexity of experimentation or simulation involved. Here, we leverage mutual information across different tasks in molecular data to address this issue. We extend an algorithm that utilizes the geometric characteristics of the encoding space, known as the Geometrically Aligned Transfer Encoder (GATE), to a multi-task setup. Thus, we connect multiple molecular tasks by aligning the curved coordinates onto locally flat coordinates, ensuring the flow of information from source tasks to support performance on target data., Comment: 7 pages, 3 figures, 2 tables

Transfer learning is a crucial technique for handling a small amount of data that is potentially related to other abundant data. However, most of the existing methods are focused on classification tasks using images and language datasets. Therefore, in order to expand the transfer learning scheme to regression tasks, we propose a novel transfer technique based on differential geometry, namely the Geometrically Aligned Transfer Encoder (GATE). In this method, we interpret the latent vectors from the model to exist on a Riemannian curved manifold. We find a proper diffeomorphism between pairs of tasks to ensure that every arbitrary point maps to a locally flat coordinate in the overlapping region, allowing the transfer of knowledge from the source to the target data. This also serves as an effective regularizer for the model to behave in extrapolation regions. In this article, we demonstrate that GATE outperforms conventional methods and exhibits stable behavior in both the latent space and extrapolation regions for various molecular graph datasets., Comment: 12+11 pages, 6+1 figures, 0+7 tables

Pretraining molecular representations from large unlabeled data is essential for molecular property prediction due to the high cost of obtaining ground-truth labels. While there exist various 2D graph-based molecular pretraining approaches, these methods struggle to show statistically significant gains in predictive performance. Recent work have thus instead proposed 3D conformer-based pretraining under the task of denoising, which led to promising results. During downstream finetuning, however, models trained with 3D conformers require accurate atom-coordinates of previously unseen molecules, which are computationally expensive to acquire at scale. In light of this limitation, we propose D&D, a self-supervised molecular representation learning framework that pretrains a 2D graph encoder by distilling representations from a 3D denoiser. With denoising followed by cross-modal knowledge distillation, our approach enjoys use of knowledge obtained from denoising as well as painless application to downstream tasks with no access to accurate conformers. Experiments on real-world molecular property prediction datasets show that the graph encoder trained via D&D can infer 3D information based on the 2D graph and shows superior performance and label-efficiency against other baselines., Comment: 16 pages, 5 figures

Graph pooling is a crucial operation for encoding hierarchical structures within graphs. Most existing graph pooling approaches formulate the problem as a node clustering task which effectively captures the graph topology. Conventional methods ask users to specify an appropriate number of clusters as a hyperparameter, then assume that all input graphs share the same number of clusters. In inductive settings where the number of clusters can vary, however, the model should be able to represent this variation in its pooling layers in order to learn suitable clusters. Thus we propose GMPool, a novel differentiable graph pooling architecture that automatically determines the appropriate number of clusters based on the input data. The main intuition involves a grouping matrix defined as a quadratic form of the pooling operator, which induces use of binary classification probabilities of pairwise combinations of nodes. GMPool obtains the pooling operator by first computing the grouping matrix, then decomposing it. Extensive evaluations on molecular property prediction tasks demonstrate that our method outperforms conventional methods., Comment: 10 pages, 3 figures

Double Field Theory suggests to view the whole massless sector of closed strings as the gravitational unity. The fundamental symmetries therein, including the $\mathbf{O}(D,D)$ covariance, can determine unambiguously how the Standard Model as well as a relativistic point particle should couple to the closed string massless sector. The theory also refines the notion of singularity. We consider the most general, spherically symmetric, asymptotically flat, static vacuum solution to ${D=4}$ Double Field Theory, which contains three free parameters and consequently generalizes the Schwarzschild geometry. Analyzing the circular geodesic of a point particle in string frame, we obtain the orbital velocity as a function of $R/(M_{\scriptscriptstyle{\infty}}G)$ which is the dimensionless radial variable normalized by mass. The rotation curve generically features a maximum and thus non-Keplerian over a finite range, while becoming asymptotically Keplerian at infinity, $R/(M_{\scriptscriptstyle{\infty}}G)\rightarrow \infty$. The adoption of the string frame rather than Einstein frame is the consequence of the fundamental symmetry principle. Our result opens up a new scheme to solve the dark matter/energy problems by modifying General Relativity at `short' range of $R/(M_{\scriptscriptstyle{\infty}}G)$., Comment: 1+36 pages, 10 figures. Final version to appear in JCAP