Search

We develop analytical tools and perform three-body simulations to investigate the orbital evolution and dynamical stability of binary planets within star clusters. Our analytical results show that the orbital stability of a planetary-mass binary against passing stars is mainly related to its orbital period. We find that critical flybys, defined as stellar encounters with energy kicks comparable to the binary binding energy, can efficiently produce a wide range of semimajor axes ($a$) and eccentricities ($e$) from a dominant population of primordially tight JuMBOs. Applying our results to the recently discovered Jupiter-Mass Binary Objects (JuMBOs) by the James Webb Space Telescope (JWST), our simulations suggest that to match the observed $\sim$9\% wide binary fraction, an initial semimajor axis of $a_0 = 10$--20~au and a density-weighted residence time of $\chi \gtrsim 10^4$~Myr~pc$^{-3}$ are favored. These results imply that the JWST JuMBOs probably formed as tight binaries near the cluster core., Comment: 9 pages, 4 figures, submitted to ApJL

Among the nearly 30,000 known near-Earth asteroids (NEAs), only tens of them possess Earth co-orbital characteristics with semi-major axes $\sim$1 au. In particular, 469219 Kamo`oalewa (2016 HO3), upcoming target of China's Tianwen-2 asteroid sampling mission, exhibits a meta-stable 1:1 mean-motion resonance with Earth. Intriguingly, recent ground-based observations show that Kamo`oalewa has spectroscopic characteristics similar to space-weathered lunar silicates, hinting at a lunar origin instead of an asteroidal one like the vast majority of NEAs. Here we use numerical simulations to demonstrate that Kamo`oalewa's physical and orbital properties are compatible with a fragment from a crater larger than 10--20 km formed on the Moon in the last few million years. The impact could have ejected sufficiently large fragments into heliocentric orbits, some of which could be transferred to Earth 1:1 resonance and persist today. This leads us to suggest the young lunar crater Giordano Bruno (22 km diameter, 1--10 Ma age) as the most likely source, linking a specific asteroid in space to its source crater on the Moon. The hypothesis will be tested by the Tianwen-2 mission when it returns a sample of Kamo`oalewa. And the upcoming NEO Surveyor mission will possibly help us to identify such a lunar-derived NEA population., Comment: 29 pages, 4 figures. Published in Nature Astronomy, 19 April 2024

Recent work has aimed to improve LLM generations by filtering out hallucinations, thereby improving the precision of the information in responses. Correctness of a long-form response, however, also depends on the recall of multiple pieces of information relevant to the question. In this paper, we introduce Atomic Self-Consistency (ASC), a technique for improving the recall of relevant information in an LLM response. ASC follows recent work, Universal Self-Consistency (USC) in using multiple stochastic samples from an LLM to improve the long-form response. Unlike USC which only focuses on selecting the best single generation, ASC picks authentic subparts from the samples and merges them into a superior composite answer. Through extensive experiments and ablations, we show that merging relevant subparts of multiple samples performs significantly better than picking a single sample. ASC demonstrates significant gains over USC on multiple factoids and open-ended QA datasets - ASQA, QAMPARI, QUEST, ELI5 with ChatGPT and Llama2. Our analysis also reveals untapped potential for enhancing long-form generations using approach of merging multiple samples., Comment: 12 pages

To enhance Large Language Models' (LLMs) reliability, calibration is essential -- the model's assessed confidence scores should align with the actual likelihood of its responses being correct. However, current confidence elicitation methods and calibration metrics typically rely on a binary true/false assessment of response correctness. This approach does not apply to long-form generation, where an answer can be partially correct. Addressing this gap, we introduce a unified calibration framework, in which both the correctness of the LLMs' responses and their associated confidence levels are treated as distributions across a range of scores. Within this framework, we develop three metrics to precisely evaluate LLM calibration and further propose two confidence elicitation methods based on self-consistency and self-evaluation. Our experiments, which include long-form QA and summarization tasks, demonstrate that larger models don't necessarily guarantee better calibration, that calibration performance is found to be metric-dependent, and that self-consistency methods excel in factoid datasets. We also find that calibration can be enhanced through techniques such as fine-tuning, integrating relevant source documents, scaling the temperature, and combining self-consistency with self-evaluation. Lastly, we showcase a practical application of our system: selecting and cascading open-source models and ChatGPT to optimize correctness given a limited API budget. This research not only challenges existing notions of LLM calibration but also offers practical methodologies for improving trustworthiness in long-form generation.

Utilizing pre-trained 2D large-scale generative models, recent works are capable of generating high-quality novel views from a single in-the-wild image. However, due to the lack of information from multiple views, these works encounter difficulties in generating controllable novel views. In this paper, we present DreamComposer, a flexible and scalable framework that can enhance existing view-aware diffusion models by injecting multi-view conditions. Specifically, DreamComposer first uses a view-aware 3D lifting module to obtain 3D representations of an object from multiple views. Then, it renders the latent features of the target view from 3D representations with the multi-view feature fusion module. Finally the target view features extracted from multi-view inputs are injected into a pre-trained diffusion model. Experiments show that DreamComposer is compatible with state-of-the-art diffusion models for zero-shot novel view synthesis, further enhancing them to generate high-fidelity novel view images with multi-view conditions, ready for controllable 3D object reconstruction and various other applications., Comment: Project Page: https://yhyang-myron.github.io/DreamComposer/

We examined the past history of the three most detached TransNeptunian Objects (TNOs) -- Sedna, 2012 VP113, and Leleakuhonua (2015 TG387) -- the three clearest members of the dynamical class known as sednoids, with high perihelia distances $q$. By integrating backward their nominal (and a set of cloned) orbits for the Solar System's age, we surprisingly find that the only time all their apsidal lines tightly cluster was 4.5 Gyr ago, at perihelion longitude $\varpi$ of $200^\circ$. This "primordial alignment" is independent of the observational biases that contribute to the current on-sky clustering in the large-semimajor axis Kuiper Belt. If future sednoid discoveries confirm these findings, this strongly argues for an initial event during the planet formation epoch which imprinted this particular apsidal orientation on the early detached TNO population. Their apsidal orientations were then subsequently modified only by the simple precession from the 4 giant planets (and weakly by the galactic tide). If other sednoids also cluster around the same primordial value, various models suggesting a still present planet in the outer Solar System would be incompatible with this alignment. We inspected two scenarios that could potentially explain the primordial alignment. First, a rogue planet model (where another massive planet raises perihelia near its own longitude until ejection) naturally produces this signature. Alternatively, a close stellar passage early in Solar System history raises perihelia, but it is poor at creating strong apsidal clustering. We show that all other known $35

The ability to derive useful information by asking clarifying questions (ACQ) is an important element of real life collaboration on reasoning tasks, such as question answering (QA). Existing natural language ACQ challenges, however, evaluate generations based on word overlap rather than the value of the information itself. Word overlap is often an inappropriate metric for question generation since many different questions could be useful in a given situation, and a single question can be phrased many different ways. Instead, we propose evaluating questions pragmatically based on the value of the information they retrieve. Here we present a definition and framework for natural language pragmatic asking of clarifying questions (PACQ), the problem of generating questions that result in answers useful for a reasoning task. We also present fact-level masking (FLM), a procedure for converting natural language datasets into self-supervised PACQ datasets by omitting particular critical facts. Finally, we generate a PACQ dataset from the HotpotQA dataset using FLM and evaluate several zero-shot language models on it. Our experiments show that current zero-shot models struggle to ask questions that retrieve useful information, as compared to human annotators. These results demonstrate an opportunity to use FLM datasets and the PACQ framework to objectively evaluate and improve question generation and other language models.

In this paper, we present TOSS, which introduces text to the task of novel view synthesis (NVS) from just a single RGB image. While Zero-1-to-3 has demonstrated impressive zero-shot open-set NVS capability, it treats NVS as a pure image-to-image translation problem. This approach suffers from the challengingly under-constrained nature of single-view NVS: the process lacks means of explicit user control and often results in implausible NVS generations. To address this limitation, TOSS uses text as high-level semantic information to constrain the NVS solution space. TOSS fine-tunes text-to-image Stable Diffusion pre-trained on large-scale text-image pairs and introduces modules specifically tailored to image and camera pose conditioning, as well as dedicated training for pose correctness and preservation of fine details. Comprehensive experiments are conducted with results showing that our proposed TOSS outperforms Zero-1-to-3 with more plausible, controllable and multiview-consistent NVS results. We further support these results with comprehensive ablations that underscore the effectiveness and potential of the introduced semantic guidance and architecture design.

The detached transneptunian objects (TNOs) are those with semimajor axes beyond the 2:1 resonance with Neptune, which are neither resonant nor scattering. Using the detached sample from the OSSOS telescopic survey, we produce the first studies of their orbital distribution based on matching the orbits and numbers of the known TNOs after accounting for survey biases. We show that the detached TNO perihelion ($q$) distribution cannot be uniform, but is instead better matched by two uniform components with a break near $q\approx40$ au. We produce parametric two-component models that are not rejectable by the OSSOS data set, and estimate that there are $36,\!000^{+12,000}_{-9,000}$ detached TNOs with absolute magnitudes $H_r < 8.66$ ($D \gtrsim 100$ km) and semimajor axes $48 < a < 250$ au (95% confidence limits). Although we believe these heuristic two-parameter models yield a correct population estimate, we then use the same methods to show that the perihelion distribution of a detached disk created by a simulated rogue planet matches the $q$ distribution even better, suggesting that the temporary presence of other planets in the early Solar System is a promising model to create today's large semimajor axis TNO population. This numerical model results in a detached TNO population estimate of $48,\!000^{+15,000}_{-12,000}$. Because this illustrates how difficult-to-detect $q>50$ au objects are likely present, we conclude that there are $(5 \pm 2)\times10^4$ dynamically detached TNOs, which are thus roughly twice as numerous as the entire transneptunian hot main belt., Comment: Accepted for publication in The Planetary Science Journal. 16 pages, 8 figures

Text-to-image diffusion models pre-trained on billions of image-text pairs have recently enabled 3D content creation by optimizing a randomly initialized differentiable 3D representation with score distillation. However, the optimization process suffers slow convergence and the resultant 3D models often exhibit two limitations: (a) quality concerns such as missing attributes and distorted shape and texture; (b) extremely low diversity comparing to text-guided image synthesis. In this paper, we show that the conflict between the 3D optimization process and uniform timestep sampling in score distillation is the main reason for these limitations. To resolve this conflict, we propose to prioritize timestep sampling with monotonically non-increasing functions, which aligns the 3D optimization process with the sampling process of diffusion model. Extensive experiments show that our simple redesign significantly improves 3D content creation with faster convergence, better quality and diversity., Comment: ICLR 2024

We present DreamWaltz, a novel framework for generating and animating complex 3D avatars given text guidance and parametric human body prior. While recent methods have shown encouraging results for text-to-3D generation of common objects, creating high-quality and animatable 3D avatars remains challenging. To create high-quality 3D avatars, DreamWaltz proposes 3D-consistent occlusion-aware Score Distillation Sampling (SDS) to optimize implicit neural representations with canonical poses. It provides view-aligned supervision via 3D-aware skeleton conditioning which enables complex avatar generation without artifacts and multiple faces. For animation, our method learns an animatable 3D avatar representation from abundant image priors of diffusion model conditioned on various poses, which could animate complex non-rigged avatars given arbitrary poses without retraining. Extensive evaluations demonstrate that DreamWaltz is an effective and robust approach for creating 3D avatars that can take on complex shapes and appearances as well as novel poses for animation. The proposed framework further enables the creation of complex scenes with diverse compositions, including avatar-avatar, avatar-object and avatar-scene interactions. See https://dreamwaltz3d.github.io/ for more vivid 3D avatar and animation results., Comment: To appear in NeurIPS 2023; Project page: https://dreamwaltz3d.github.io/

Given the success with in-context learning of large pre-trained language models, we introduce in-context learning distillation to transfer in-context few-shot learning ability from large models to smaller models. We propose to combine in-context learning objectives with language modeling objectives to distill both the ability to read in-context examples and task knowledge to the smaller models. We perform in-context learning distillation under two different few-shot learning paradigms: Meta In-context Tuning (Meta-ICT) and Multitask In-context Tuning (Multitask-ICT). Multitask-ICT performs better on multitask few-shot learning but also requires more computation than Meta-ICT. Our method shows consistent improvements for both Meta-ICT and Multitask-ICT on two benchmarks: LAMA and CrossFit. Our extensive experiments and analysis reveal that in-context learning objectives and language modeling objectives are complementary under the Multitask-ICT paradigm. In-context learning objectives achieve the best performance when combined with language modeling objectives.

This work presents two astonishing findings on neural networks learned for large-scale image classification. 1) Given a well-trained model, the logits predicted for some category can be directly obtained by linearly combining the predictions of a few other categories, which we call \textbf{neural dependency}. 2) Neural dependencies exist not only within a single model, but even between two independently learned models, regardless of their architectures. Towards a theoretical analysis of such phenomena, we demonstrate that identifying neural dependencies is equivalent to solving the Covariance Lasso (CovLasso) regression problem proposed in this paper. Through investigating the properties of the problem solution, we confirm that neural dependency is guaranteed by a redundant logit covariance matrix, which condition is easily met given massive categories, and that neural dependency is highly sparse, implying that one category correlates to only a few others. We further empirically show the potential of neural dependencies in understanding internal data correlations, generalizing models to unseen categories, and improving model robustness with a dependency-derived regularizer. Code for this work will be made publicly available.

The orbital distribution of transneptunian objects (TNOs) in the distant Kuiper Belt (with semimajor axes beyond the 2:1 resonance, roughly $a$=50-100 au) provides constraints on the dynamical history of the outer solar system. Recent studies show two striking features of this region: 1) a very large population of objects in distant mean-motion resonances with Neptune, and 2) the existence of a substantial detached population (non-resonant objects largely decoupled from Neptune). Neptune migration models are able to implant some resonant and detached objects during the planet migration era, but many fail to match a variety of aspects of the orbital distribution. In this work, we report simulations carried out using an improved version of the GPU-based code GLISSE, following 100,000 test particles per simulation in parallel while handling their planetary close encounters. We demonstrate for the first time that a 2 Earth-mass rogue planet temporarily present during planet formation can abundantly populate both the distant resonances and the detached populations, surprisingly even without planetary migration. We show how weak encounters with the rogue greatly increase the efficiency of filling the resonances, while also dislodging TNOs out of resonance once they reach high perihelia. The rogue's secular gravitational influence simultaneously generates numerous detached objects observed at all semimajor axes. These results suggest that the early presence of additional planet(s) reproduces the observed TNO orbital structure in the distant Kuiper Belt., Comment: 14 pages, 5 figures. accepted for publication in ApJ Letter. For associated animated movies, see https://yukunhuang.com/

There has been growing research interest in using deep learning based method to achieve fully automated segmentation of lesion in Positron emission tomography computed tomography(PET CT) scans for the prognosis of various cancers. Recent advances in the medical image segmentation shows the nnUNET is feasible for diverse tasks. However, lesion segmentation in the PET images is not straightforward, because lesion and physiological uptake has similar distribution patterns. The Distinction of them requires extra structural information in the CT images. The present paper introduces a nnUNet based method for the lesion segmentation task. The proposed model is designed on the basis of the joint 2D and 3D nnUNET architecture to predict lesions across the whole body. It allows for automated segmentation of potential lesions. We evaluate the proposed method in the context of AutoPet Challenge, which measures the lesion segmentation performance in the metrics of dice score, false-positive volume and false-negative volume.

The rank of neural networks measures information flowing across layers. It is an instance of a key structural condition that applies across broad domains of machine learning. In particular, the assumption of low-rank feature representations leads to algorithmic developments in many architectures. For neural networks, however, the intrinsic mechanism that yields low-rank structures remains vague and unclear. To fill this gap, we perform a rigorous study on the behavior of network rank, focusing particularly on the notion of rank deficiency. We theoretically establish a universal monotonic decreasing property of network rank from the basic rules of differential and algebraic composition, and uncover rank deficiency of network blocks and deep function coupling. By virtue of our numerical tools, we provide the first empirical analysis of the per-layer behavior of network rank in practical settings, i.e., ResNets, deep MLPs, and Transformers on ImageNet. These empirical results are in direct accord with our theory. Furthermore, we reveal a novel phenomenon of independence deficit caused by the rank deficiency of deep networks, where classification confidence of a given category can be linearly decided by the confidence of a handful of other categories. The theoretical results of this work, together with the empirical findings, may advance understanding of the inherent principles of deep neural networks., Comment: 31 pages, 12 figures

Prompt tuning (PT) is an effective approach to adapting pre-trained language models to downstream tasks. Without a good initialization, prompt tuning doesn't perform well under few-shot settings. So pre-trained prompt tuning (PPT) is proposed to initialize prompts by leveraging pre-training data. We propose MetaPT (Meta-learned Prompt Tuning) to further improve PPT's initialization by considering latent structure within the pre-training data. Specifically, we introduce the structure by first clustering pre-training data into different auxiliary tasks with unsupervised methods. Then we use these tasks to pre-train prompts with a meta-learning algorithm. Such a process can make prompts learn a better initialization by discovering commonalities among these auxiliary tasks. We evaluate our method on seven downstream tasks. Our MetaPT achieves better and more stable performance than the state-of-the-art method.

There is a complex inclination structure present in the transneptunian object (TNO) orbital distribution in the main classical belt region (between orbital semimajor axes of 39 and 48 au). The long-term gravitational effects of the giant planets make TNO orbits precess, but non-resonant objects maintain a nearly constant 'free' inclination ($I_\text{free}$) with respect to a local forced precession pole. Because of the likely cosmogonic importance of the distribution of this quantity, we tabulate free inclinations for all main-belt TNOs, each individually computed using barycentric orbital elements with respect to each object's local forcing pole. We show that the simplest method, based on the Laplace-Lagrange secular theory, is unable to give correct forcing poles for objects near the $\nu_{18}$ secular resonance, resulting in poorly conserved $I_\text{free}$ values in much of the main belt. We thus instead implemented an averaged Hamiltonian to obtain the expected nodal precession for each TNO, yielding significantly more accurate free inclinations for non-resonant objects. For the vast majority (96\%) of classical belt TNOs, these $I_\text{free}$ values are conserved to $<1^\circ$ over 4 Gyr numerical simulations, demonstrating the advantage of using this well-conserved quantity in studies of the TNO population and its primordial inclination profile; our computed distributions only reinforce the idea of a very co-planar surviving 'cold' primordial population, overlain by a large $I$-width implanted 'hot' population., Comment: 15 pages, 6 figures, accepted for publication in ApJS. Data downloadable in https://yukunhuang.com

Consumer Event-Cause Extraction, the task aimed at extracting the potential causes behind certain events in the text, has gained much attention in recent years due to its wide applications. The ICDM 2020 conference sets up an evaluation competition that aims to extract events and the causes of the extracted events with a specified subject (a brand or product). In this task, we mainly focus on how to construct an end-to-end model, and extract multiple event types and event-causes simultaneously. To this end, we introduce a fresh perspective to revisit the relational event-cause extraction task and propose a novel sequence tagging framework, instead of extracting event types and events-causes separately. Experiments show our framework outperforms baseline methods even when its encoder module uses an initialized pre-trained BERT encoder, showing the power of the new tagging framework. In this competition, our team achieved 1st place in the first stage leaderboard, and 3rd place in the final stage leaderboard.

Previous work has demonstrated orbital stability for 100 Myr of initially near-circular and coplanar small bodies in a region termed the 'Earth-Mars belt' from 1.08 au $< a <$ 1.28 au. Via numerical integration of 3000 particles, we studied orbits from 1.04-1.30 au for the age of the Solar system. We show that on this time scale, except for a few locations where mean-motion resonances with Earth affect stability, only a narrower `Earth-Mars belt' covering $a\sim(1.09, 1.17)$ au, $e<0.04$, and $I<1^\circ$ has over half of the initial orbits survive for 4.5 Gyr. In addition to mean-motion resonances, we are able to see how the $\nu_3$, $\nu_4$, and $\nu_6$ secular resonances contribute to long-term instability in the outer (1.17-1.30 au) region on Gyr time scales. We show that all of the (rather small) near-Earth objects (NEOs) in or close to the Earth-Mars belt appear to be consistent with recently arrived transient objects by comparing to a NEO steady-state model. Given the $<200$ m scale of these NEOs, we estimated the Yarkovsky drift rates in semimajor axis, and use these to estimate that primordial asteroids with a diameter of 100 km or larger in the Earth-Mars belt would likely survive. We conclude that only a few 100-km sized asteroids could have been present in the belt's region at the end of the terrestrial planet formation., Comment: 8 pages, 4 figures. Accepted for publication in MNRAS

Person re-identification (Re-ID) in real-world scenarios usually suffers from various degradation factors, e.g., low-resolution, weak illumination, blurring and adverse weather. On the one hand, these degradations lead to severe discriminative information loss, which significantly obstructs identity representation learning; on the other hand, the feature mismatch problem caused by low-level visual variations greatly reduces retrieval performance. An intuitive solution to this problem is to utilize low-level image restoration methods to improve the image quality. However, existing restoration methods cannot directly serve to real-world Re-ID due to various limitations, e.g., the requirements of reference samples, domain gap between synthesis and reality, and incompatibility between low-level and high-level methods. In this paper, to solve the above problem, we propose a degradation invariance learning framework for real-world person Re-ID. By introducing a self-supervised disentangled representation learning strategy, our method is able to simultaneously extract identity-related robust features and remove real-world degradations without extra supervision. We use low-resolution images as the main demonstration, and experiments show that our approach is able to achieve state-of-the-art performance on several Re-ID benchmarks. In addition, our framework can be easily extended to other real-world degradation factors, such as weak illumination, with only a few modifications., Comment: To appear in CVPR2020