Search

We develop the pole-skipping structure in de Sitter (dS) spacetime and find that their leading frequencies satisfy the relation $\omega_{dS}=i2\pi T_{dS}(1-s)$ with $T_{dS}$ denotes $T_{dS}=1/2\pi L$ and $s$ spin. The positions of pole-skipping in the two-dimensional dS spacetime for the scalar field of spin-0 and the fermionic field of spin-$\frac{1}{2}$ correspond one-to-one with those in the infinite temperature limit of double-scaled Sachdev-Ye-Kitaev model (DSSYK$_\infty$). The Green's functions of these two systems have similar forms at the pole-skipping points. This provides a numerical correspondence for the duality between dS$_2$ and the DSSYK$_\infty$.

In this work, we discuss evaluating video foundation models in a fair and robust manner. Unlike language or image foundation models, many video foundation models are evaluated with differing parameters (such as sampling rate, number of frames, pretraining steps, etc.), making fair and robust comparisons challenging. Therefore, we present a carefully designed evaluation framework for measuring two core capabilities of video comprehension: appearance and motion understanding. Our findings reveal that existing video foundation models, whether text-supervised like UMT or InternVideo2, or self-supervised like V-JEPA, exhibit limitations in at least one of these capabilities. As an alternative, we introduce TWLV-I, a new video foundation model that constructs robust visual representations for both motion- and appearance-based videos. Based on the average top-1 accuracy of linear probing on five action recognition benchmarks, pretrained only on publicly accessible datasets, our model shows a 4.6%p improvement compared to V-JEPA (ViT-L) and a 7.7%p improvement compared to UMT (ViT-L). Even when compared to much larger models, our model demonstrates a 7.2%p improvement compared to DFN (ViT-H), a 2.7%p improvement compared to V-JEPA (ViT-H) and a 2.8%p improvement compared to InternVideo2 (ViT-g). We provide embedding vectors obtained by TWLV-I from videos of several commonly used video benchmarks, along with evaluation source code that can directly utilize these embeddings. The code is available at https://github.com/twelvelabs-io/video-embeddings-evaluation-framework., Comment: 17 pages; Twelve Labs Technical Report

This paper introduces mhGPT, a lightweight generative pre-trained transformer trained on mental health-related social media and PubMed articles. Fine-tuned for specific mental health tasks, mhGPT was evaluated under limited hardware constraints and compared with state-of-the-art models like MentaLLaMA and Gemma. Despite having only 1.98 billion parameters and using just 5% of the dataset, mhGPT outperformed larger models and matched the performance of models trained on significantly more data. The key contributions include integrating diverse mental health data, creating a custom tokenizer, and optimizing a smaller architecture for low-resource settings. This research could advance AI-driven mental health care, especially in areas with limited computing power.

The explosive arrival of OpenAI's ChatGPT has fueled the globalization of large language model (LLM), which consists of billions of pretrained parameters that embodies the aspects of syntax and semantics. HyperAccel introduces latency processing unit (LPU), a latency-optimized and highly scalable processor architecture for the acceleration of LLM inference. LPU perfectly balances the memory bandwidth and compute logic with streamlined dataflow to maximize performance and efficiency. LPU is equipped with expandable synchronization link (ESL) that hides data synchronization latency between multiple LPUs. HyperDex complements LPU as an intuitive software framework to run LLM applications. LPU achieves 1.25 ms/token and 20.9 ms/token for 1.3B and 66B model, respectively, which is 2.09x and 1.37x faster than the GPU. LPU, synthesized using Samsung 4nm process, has total area of 0.824 mm2 and power consumption of 284.31 mW. LPU-based servers achieve 1.33x and 1.32x energy efficiency over NVIDIA H100 and L4 servers, respectively.

The "Battlefield" online game is well-known for its large-scale multiplayer capabilities and unique gaming features, including various vehicle controls. However, these features make the game a major target for cheating, significantly detracting from the gaming experience. This study analyzes user behavior in cheating play in the popular online game, the "Battlefield", using statistical methods. We aim to provide comprehensive insights into cheating players through an extensive analysis of over 44,000 reported cheating incidents collected via the "Game-tools API". Our methodology includes detailed statistical analyses such as calculating basic statistics of key variables, correlation analysis, and visualizations using histograms, box plots, and scatter plots. Our findings emphasize the importance of adaptive, data-driven approaches to prevent cheating plays in online games., Comment: 12 pages, 5 figures, 2 tables; accepted to the 25th World Conference on Information Security Applications (WISA 2024)

Krylov complexity has recently emerged as a new paradigm to characterize quantum chaos in many-body systems. However, which features of Krylov complexity are prerogative of quantum chaotic systems and how they relate to more standard probes, such as spectral statistics or out-of-time-order correlators (OTOCs), remain open questions. Recent insights have revealed that in quantum chaotic systems Krylov state complexity exhibits a distinct peak during time evolution before settling into a well-understood late-time plateau. In this work, we propose that this Krylov complexity peak (KCP) is a hallmark of quantum chaotic systems and suggest that its height could serve as an `order parameter' for quantum chaos. We demonstrate that the KCP effectively identifies chaotic-integrable transitions in the mass-deformed Sachdev-Ye-Kitaev model at both infinite and finite temperature, aligning with results from spectral statistics and OTOCs. Our findings offer a new diagnostic tool for quantum chaos that is operator-independent, potentially leading to more `universal' insights and a deeper understanding of general properties in quantum chaotic systems., Comment: v1: 6 pages, 3 figures, v2: references added, minor changes, supplemental material included

With large language models (LLMs), conversational search engines shift how users retrieve information from the web by enabling natural conversations to express their search intents over multiple turns. Users' natural conversation embodies rich but implicit signals of users' search intents and evaluation of search results to understand user experience with the system. However, it is underexplored how and why users ask follow-up queries to continue conversations with conversational search engines and how the follow-up queries signal users' satisfaction. From qualitative analysis of 250 conversational turns from an in-lab user evaluation of Naver Cue:, a commercial conversational search engine, we propose a taxonomy of 18 users' follow-up query patterns from conversational search, comprising two major axes: (1) users' motivations behind continuing conversations (N = 7) and (2) actions of follow-up queries (N = 11). Compared to the existing literature on query reformulations, we uncovered a new set of motivations and actions behind follow-up queries, including asking for subjective opinions or providing natural language feedback on the engine's responses. To analyze conversational search logs with our taxonomy in a scalable and efficient manner, we built an LLM-powered classifier (73% accuracy). With our classifier, we analyzed 2,061 conversational tuples collected from real-world usage logs of Cue: and examined how the conversation patterns from our taxonomy correlates with satisfaction. Our initial findings suggest some signals of dissatisfactions, such as Clarifying Queries, Excluding Condition, and Substituting Condition with follow-up queries. We envision our approach could contribute to automated evaluation of conversation search experience by providing satisfaction signals and grounds for realistic user simulations., Comment: Accepted to LLM4Eval @ SIGIR 2024 - The First Workshop on Large Language Models (LLMs) for Evaluation in Information Retrieval

It has become common practice now to use random initialization schemes, rather than the pre-trained embeddings, when training transformer based models from scratch. Indeed, we find that pre-trained word embeddings from GloVe, and some sub-word embeddings extracted from language models such as T5 and mT5 fare much worse compared to random initialization. This is counter-intuitive given the well-known representational and transfer-learning advantages of pre-training. Interestingly, we also find that BERT and mBERT embeddings fare better than random initialization, showing the advantages of pre-trained representations. In this work, we posit two potential factors that contribute to these mixed results: the model sensitivity to parameter distribution and the embedding interactions with position encodings. We observe that pre-trained GloVe, T5, and mT5 embeddings have a wider distribution of values. As argued in the initialization studies, such large value initializations can lead to poor training because of saturated outputs. Further, the larger embedding values can, in effect, absorb the smaller position encoding values when added together, thus losing position information. Standardizing the pre-trained embeddings to a narrow range (e.g. as prescribed by Xavier) leads to substantial gains for Glove, T5, and mT5 embeddings. On the other hand, BERT pre-trained embeddings, while larger, are still relatively closer to Xavier initialization range which may allow it to effectively transfer the pre-trained knowledge.

Class-Incremental Semantic Segmentation(CISS) aims to learn new classes without forgetting the old ones, using only the labels of the new classes. To achieve this, two popular strategies are employed: 1) pseudo-labeling and knowledge distillation to preserve prior knowledge; and 2) background weight transfer, which leverages the broad coverage of background in learning new classes by transferring background weight to the new class classifier. However, the first strategy heavily relies on the old model in detecting old classes while undetected pixels are regarded as the background, thereby leading to the background shift towards the old classes(i.e., misclassification of old class as background). Additionally, in the case of the second approach, initializing the new class classifier with background knowledge triggers a similar background shift issue, but towards the new classes. To address these issues, we propose a background-class separation framework for CISS. To begin with, selective pseudo-labeling and adaptive feature distillation are to distill only trustworthy past knowledge. On the other hand, we encourage the separation between the background and new classes with a novel orthogonal objective along with label-guided output distillation. Our state-of-the-art results validate the effectiveness of these proposed methods., Comment: Accepted to ECCV 2024. Code is available at http://github.com/RoadoneP/ECCV2024_MBS

The flatfish is a major farmed species consumed globally in large quantities. However, due to the densely populated farming environment, flatfish are susceptible to injuries and diseases, making early disease detection crucial. Traditionally, diseases were detected through visual inspection, but observing large numbers of fish is challenging. Automated approaches based on deep learning technologies have been widely used, to address this problem, but accurate detection remains difficult due to the diversity of the fish and the lack of the fish disease dataset. In this study, augments fish disease images using generative adversarial networks and image harmonization methods. Next, disease detectors are trained separately for three body parts (head, fins, and body) to address individual diseases properly. In addition, a flatfish disease image dataset called \texttt{FlatIMG} is created and verified on the dataset using the proposed methods. A flash salmon disease dataset is also tested to validate the generalizability of the proposed methods. The results achieved 12\% higher performance than the baseline framework. This study is the first attempt to create a large-scale flatfish disease image dataset and propose an effective disease detection framework. Automatic disease monitoring could be achieved in farming environments based on the proposed methods and dataset., Comment: 16 page, 13 figures, 4 tables

Interacting individuals in complex systems often give rise to coherent motion exhibiting coordinated global structures. Such phenomena are ubiquitously observed in nature, from cell migration, bacterial swarms, animal and insect groups, and even human societies. Primary mechanisms responsible for the emergence of collective behavior have been extensively identified, including local alignments based on average or relative velocity, non-local pairwise repulsive-attractive interactions such as distance-based potentials, interplay between local and non-local interactions, and cognitive-based inhomogeneous interactions. However, discovering how to adapt these mechanisms to modulate emergent behaviours remains elusive. Here, we demonstrate that it is possible to generate coordinated structures in collective behavior at desired moments with intended global patterns by fine-tuning an inter-agent interaction rule. Our strategy employs deep neural networks, obeying the laws of dynamics, to find interaction rules that command desired collective structures. The decomposition of interaction rules into distancing and aligning forces, expressed by polynomial series, facilitates the training of neural networks to propose desired interaction models. Presented examples include altering the mean radius and size of clusters in vortical swarms, timing of transitions from random to ordered states, and continuously shifting between typical modes of collective motions. This strategy can even be leveraged to superimpose collective modes, resulting in hitherto unexplored but highly practical hybrid collective patterns, such as protective security formations. Our findings reveal innovative strategies for creating and controlling collective motion, paving the way for new applications in robotic swarm operations, active matter organisation, and for the uncovering of obscure interaction rules in biological systems.

We consider functional linear regression models where functional outcomes are associated with scalar predictors by coefficient functions with shape constraints, such as monotonicity and convexity, that apply to sub-domains of interest. To validate the partial shape constraints, we propose testing a composite hypothesis of linear functional constraints on regression coefficients. Our approach employs kernel- and spline-based methods within a unified inferential framework, evaluating the statistical significance of the hypothesis by measuring an $L^2$-distance between constrained and unconstrained model fits. In the theoretical study of large-sample analysis under mild conditions, we show that both methods achieve the standard rate of convergence observed in the nonparametric estimation literature. Through numerical experiments of finite-sample analysis, we demonstrate that the type I error rate keeps the significance level as specified across various scenarios and that the power increases with sample size, confirming the consistency of the test procedure under both estimation methods. Our theoretical and numerical results provide researchers the flexibility to choose a method based on computational preference. The practicality of partial shape-constrained inference is illustrated by two data applications: one involving clinical trials of NeuroBloc in type A-resistant cervical dystonia and the other with the National Institute of Mental Health Schizophrenia Study., Comment: 30 pages, 7 figures

Quantization is a promising solution for deploying large-scale language models (LLMs) on resource-constrained devices. Existing quantization approaches, however, rely on gradient-based optimization, regardless of it being post-training quantization (PTQ) or quantization-aware training (QAT), which becomes problematic for hyper-scale LLMs with billions of parameters. This overhead can be alleviated via recently proposed backpropagation-free PTQ methods; however, their performance is somewhat limited by their lack of consideration of inter-layer dependencies. In this paper, we thus propose a novel PTQ algorithm that considers inter-layer dependencies without relying on backpropagation. The fundamental concept involved is the development of attention-aware Hessian matrices, which facilitates the consideration of inter-layer dependencies within the attention module. Extensive experiments demonstrate that the proposed algorithm significantly outperforms conventional PTQ methods, particularly for low bit-widths., Comment: 20 pages, under review

We report on investigations of mode coupling between rotating spokes during the onset of the breathing oscillation. Demonstrating the existence of nonlinear coupling between the sporadic spokes and the breathing oscillation, we suggest the oscillating azimuthal electric field as the energy source for additional ionization within the plasma. Our results indicate that intermittent three-wave coupling is a possible mechanism for triggering low-frequency breathing oscillations in partially magnetized cross-field plasma.

We investigate the bulk reconstruction of AdS black hole spacetime emergent from quantum entanglement within a machine learning framework. Utilizing neural ordinary differential equations alongside Monte-Carlo integration, we develop a method tailored for continuous training functions to extract the general isotropic bulk metric from entanglement entropy data. To validate our approach, we first apply our machine learning algorithm to holographic entanglement entropy data derived from the Gubser-Rocha and superconductor models, which serve as representative models of strongly coupled matters in holography. Our algorithm successfully extracts the corresponding bulk metrics from these data. Additionally, we extend our methodology to many-body systems by employing entanglement entropy data from a fermionic tight-binding chain at half filling, exemplifying critical one-dimensional systems, and derive the associated bulk metric. We find that the metrics for a tight-binding chain and the Gubser-Rocha model are similar. We speculate this similarity is due to the metallic property of these models., Comment: 44 pages, 14 figures

The significance of mass spectrometry lies in its unparalleled ability to accurately identify and quantify molecules in complex samples, providing invaluable insights into molecular structures and interactions. Here, we leverage diamond nanostructures as highly sensitive mass sensors by utilizing a self-excitation mechanism under an electron beam in a conventional scanning electron microscope (SEM). The diamond molecular balance (DMB) exhibits an exceptional mass resolution of 0.36 MDa, based on its outstanding mechanical quality factor and frequency stability, along with an extensive dynamic range from MDa to TDa. This positions the DMB at the forefront of molecular balances operating at room temperature. Notably, the DMB demonstrates its ability to measure the mass of a single bacteriophage T4 by precisely locating the analyte on the device. These findings highlight the groundbreaking potential of the DMB as a revolutionary tool for mass spectrometry at room temperature., Comment: 16 pages, 4 figures

We present Diffusion Model Patching (DMP), a simple method to boost the performance of pre-trained diffusion models that have already reached convergence, with a negligible increase in parameters. DMP inserts a small, learnable set of prompts into the model's input space while keeping the original model frozen. The effectiveness of DMP is not merely due to the addition of parameters but stems from its dynamic gating mechanism, which selects and combines a subset of learnable prompts at every step of the generative process (e.g., reverse denoising steps). This strategy, which we term "mixture-of-prompts", enables the model to draw on the distinct expertise of each prompt, essentially "patching" the model's functionality at every step with minimal yet specialized parameters. Uniquely, DMP enhances the model by further training on the same dataset on which it was originally trained, even in a scenario where significant improvements are typically not expected due to model convergence. Experiments show that DMP significantly enhances the converged FID of DiT-L/2 on FFHQ 256x256 by 10.38%, achieved with only a 1.43% parameter increase and 50K additional training iterations., Comment: Project page: https://sangminwoo.github.io/DMP/

A single photodetector with tunable detection wavelengths and polarization sensitivity can potentially be harnessed for diverse optical applications ranging from imaging and sensing to telecommunications. Such a device will require the combination of multiple material systems with different structures, bandgaps, and photoelectrical responses, which is extremely difficult to engineer using traditional epitaxial films. Here, we develop a multi-functional and high-performance photosensor using all van der Waals materials. The device features a gate-tunable spectral response that is switchable between near-infrared/visible and short-/mid-wave infrared, as well as broadband operation, at room temperature. The linear polarization sensitivity in the telecommunications O-band can also be directly modulated between horizontal, vertical, and nonpolarizing modes. These effects originate from the balance of photocurrent generation in two of the active layers that can be manipulated by an electric field. The photodetector features high detectivity (>109 cmHz1/2W-1) together with fast operation speed (~ 1 MHz) and can be further exploited for dual visible and infrared imaging., Comment: 23 pages,16 figures

QCD gives rise to a rich spectrum of excited baryon states. Understanding their internal structure is important for many areas of nuclear physics, such as nuclear forces, dense matter, and neutrino-nucleus interactions. Generalized parton distributions (GPDs) are an established tool for characterizing the QCD structure of the ground-state nucleon. They are used to create 3D tomographic images of the quark/gluon structure and quantify the mechanical properties such as the distribution of mass, angular momentum and forces in the system. Transition GPDs extend these concepts to $N \rightarrow N^\ast$ transitions and can be used to characterize the 3D structure and mechanical properties of baryon resonances. They can be probed in high-momentum-transfer exclusive electroproduction processes with resonance transitions $e + N \rightarrow e' + M + N^\ast$, such as deeply-virtual Compton scattering ($M = \gamma$) or meson production ($M = \pi, K$, $etc.$), and in related photon/hadron-induced processes. This White Paper describes a research program aiming to explore baryon resonance structure with transition GPDs. This includes the properties and interpretation of the transition GPDs, theoretical methods for structures and processes, first experimental results from JLab 12 GeV, future measurements with existing and planned facilities (JLab detector and energy upgrades, COMPASS/AMBER, EIC, EicC, J-PARC, LHC ultraperihperal collisions), and the theoretical and experimental developments needed to realize this program.

We explore spread and spectral complexity in quantum systems that exhibit a transition from integrability to chaos, namely the mixed-field Ising model and the next-to-nearest-neighbor deformation of the Heisenberg XXZ spin chain. We corroborate the observation that the presence of a peak in spread complexity before its saturation, is a characteristic feature in chaotic systems. We find that, in general, the saturation value of spread complexity post-peak depends not only on the spectral statistics of the Hamiltonian, but also on the specific state. However, there appears to be a maximal universal bound determined by the symmetries and dimension of the Hamiltonian, which is realized by the thermofield double state (TFD) at infinite temperature. We also find that the time scales at which the spread complexity and spectral form factor change their behaviour agree with each other and are independent of the chaotic properties of the systems. In the case of spectral complexity, we identify that the key factor determining its saturation value and timescale in chaotic systems is given by minimum energy difference in the theory's spectrum. This explains observations made in the literature regarding its earlier saturation in chaotic systems compared to their integrable counterparts. We conclude by discussing the properties of the TFD which, we conjecture, make it suitable for probing signatures of chaos in quantum many-body systems., Comment: v1: 35 pages, 18 figures, v2: references added, minor changes

Session-based recommendation (SBR) aims to predict the following item a user will interact with during an ongoing session. Most existing SBR models focus on designing sophisticated neural-based encoders to learn a session representation, capturing the relationship among session items. However, they tend to focus on the last item, neglecting diverse user intents that may exist within a session. This limitation leads to significant performance drops, especially for longer sessions. To address this issue, we propose a novel SBR model, called Multi-intent-aware Session-based Recommendation Model (MiaSRec). It adopts frequency embedding vectors indicating the item frequency in session to enhance the information about repeated items. MiaSRec represents various user intents by deriving multiple session representations centered on each item and dynamically selecting the important ones. Extensive experimental results show that MiaSRec outperforms existing state-of-the-art SBR models on six datasets, particularly those with longer average session length, achieving up to 6.27% and 24.56% gains for MRR@20 and Recall@20. Our code is available at https://github.com/jin530/MiaSRec., Comment: SIGIR 2024. 5 pages

We consider the bending of light around a compact astrophysical object with both the electric field and the magnetic field in Einstein-Born-Infeld theory. From the null geodesic of a light ray passing a massive object with electric charge and magnetic dipole, the effective metric was obtained from the light cone condition reflecting the nonlinear electromagnetic effects. We found the asymptotic form of the effective metric up to the first order in gravitational constant $G$ and Born-Infeld parameter $1/\beta^2$ on the equatorial plane. Then we compute the bending angle of light from the geodesic equation. The result includes particular cases where only one type of field is present taking the appropriate limits., Comment: 11 pages, 1 figure

This technical report introduces Pegasus-1, a multimodal language model specialized in video content understanding and interaction through natural language. Pegasus-1 is designed to address the unique challenges posed by video data, such as interpreting spatiotemporal information, to offer nuanced video content comprehension across various lengths. This technical report overviews Pegasus-1's architecture, training strategies, and its performance in benchmarks on video conversation, zero-shot video question answering, and video summarization. We also explore qualitative characteristics of Pegasus-1 , demonstrating its capabilities as well as its limitations, in order to provide readers a balanced view of its current state and its future direction.

Cardiologists use electrocardiograms (ECG) for the detection of arrhythmias. However, continuous monitoring of ECG signals to detect cardiac abnormal-ities requires significant time and human resources. As a result, several deep learning studies have been conducted in advance for the automatic detection of arrhythmia. These models show relatively high performance in supervised learning, but are not applicable in cases with few training examples. This is because abnormal ECG data is scarce compared to normal data in most real-world clinical settings. Therefore, in this study, GAN-based anomaly detec-tion, i.e., unsupervised learning, was employed to address the issue of data imbalance. This paper focuses on detecting abnormal signals in electrocardi-ograms (ECGs) using only labels from normal signals as training data. In-spired by self-supervised vision transformers, which learn by dividing images into patches, and masked auto-encoders, known for their effectiveness in patch reconstruction and solving information redundancy, we introduce the ECG Heartbeat Anomaly Detection model, EB-GAME. EB-GAME was trained and validated on the MIT-BIH Arrhythmia Dataset, where it achieved state-of-the-art performance on this benchmark.

We introduce HyperCLOVA X, a family of large language models (LLMs) tailored to the Korean language and culture, along with competitive capabilities in English, math, and coding. HyperCLOVA X was trained on a balanced mix of Korean, English, and code data, followed by instruction-tuning with high-quality human-annotated datasets while abiding by strict safety guidelines reflecting our commitment to responsible AI. The model is evaluated across various benchmarks, including comprehensive reasoning, knowledge, commonsense, factuality, coding, math, chatting, instruction-following, and harmlessness, in both Korean and English. HyperCLOVA X exhibits strong reasoning capabilities in Korean backed by a deep understanding of the language and cultural nuances. Further analysis of the inherent bilingual nature and its extension to multilingualism highlights the model's cross-lingual proficiency and strong generalization ability to untargeted languages, including machine translation between several language pairs and cross-lingual inference tasks. We believe that HyperCLOVA X can provide helpful guidance for regions or countries in developing their sovereign LLMs., Comment: 44 pages; updated authors list and fixed author names

Telecom C-band single photons exhibit the lowest attenuation in optical fibers, enabling long-haul quantum-secured communication. However, efficient coupling with optical fibers is crucial for these single photons to be effective carriers in long-distance transmission. In this work, we demonstrate an efficient fiber-coupled single photon source at the telecom C-band using InAs/InP quantum dots coupled to a tapered nanobeam. The tapered nanobeam structure facilitates directional emission that is mode-matched to a lensed fiber, resulting in a collection efficiency of up to 65% from the nanobeam to a single-mode fiber. Using this approach, we demonstrate single photon count rates of 575 $\pm$ 5 Kcps and a single photon purity of $g^2$ (0) = 0.015 $\pm$ 0.003. Additionally, we demonstrate Hong-Ou Mandel interference from the emitted photons with a visibility of 0.84 $\pm$ 0.06. From these measurements, we determine a photon coherence time of 450 $\pm$ 20 ps, a factor of just 8.3 away from the lifetime limit. This work represents an important step towards the development of telecom C-band single-photon sources emitting bright, pure, and indistinguishable photons, which are necessary to realize fiber-based long-distance quantum networks

Diffusion-based generative models have emerged as powerful tools in the realm of generative modeling. Despite extensive research on denoising across various timesteps and noise levels, a conflict persists regarding the relative difficulties of the denoising tasks. While various studies argue that lower timesteps present more challenging tasks, others contend that higher timesteps are more difficult. To address this conflict, our study undertakes a comprehensive examination of task difficulties, focusing on convergence behavior and changes in relative entropy between consecutive probability distributions across timesteps. Our observational study reveals that denoising at earlier timesteps poses challenges characterized by slower convergence and higher relative entropy, indicating increased task difficulty at these lower timesteps. Building on these observations, we introduce an easy-to-hard learning scheme, drawing from curriculum learning, to enhance the training process of diffusion models. By organizing timesteps or noise levels into clusters and training models with ascending orders of difficulty, we facilitate an order-aware training regime, progressing from easier to harder denoising tasks, thereby deviating from the conventional approach of training diffusion models simultaneously across all timesteps. Our approach leads to improved performance and faster convergence by leveraging benefits of curriculum learning, while maintaining orthogonality with existing improvements in diffusion training techniques. We validate these advantages through comprehensive experiments in image generation tasks, including unconditional, class-conditional, and text-to-image generation.

We present BEHAVIOR-1K, a comprehensive simulation benchmark for human-centered robotics. BEHAVIOR-1K includes two components, guided and motivated by the results of an extensive survey on "what do you want robots to do for you?". The first is the definition of 1,000 everyday activities, grounded in 50 scenes (houses, gardens, restaurants, offices, etc.) with more than 9,000 objects annotated with rich physical and semantic properties. The second is OMNIGIBSON, a novel simulation environment that supports these activities via realistic physics simulation and rendering of rigid bodies, deformable bodies, and liquids. Our experiments indicate that the activities in BEHAVIOR-1K are long-horizon and dependent on complex manipulation skills, both of which remain a challenge for even state-of-the-art robot learning solutions. To calibrate the simulation-to-reality gap of BEHAVIOR-1K, we provide an initial study on transferring solutions learned with a mobile manipulator in a simulated apartment to its real-world counterpart. We hope that BEHAVIOR-1K's human-grounded nature, diversity, and realism make it valuable for embodied AI and robot learning research. Project website: https://behavior.stanford.edu., Comment: A preliminary version was published at 6th Conference on Robot Learning (CoRL 2022)

Diffusion models have achieved remarkable success across a range of generative tasks. Recent efforts to enhance diffusion model architectures have reimagined them as a form of multi-task learning, where each task corresponds to a denoising task at a specific noise level. While these efforts have focused on parameter isolation and task routing, they fall short of capturing detailed inter-task relationships and risk losing semantic information, respectively. In response, we introduce Switch Diffusion Transformer (Switch-DiT), which establishes inter-task relationships between conflicting tasks without compromising semantic information. To achieve this, we employ a sparse mixture-of-experts within each transformer block to utilize semantic information and facilitate handling conflicts in tasks through parameter isolation. Additionally, we propose a diffusion prior loss, encouraging similar tasks to share their denoising paths while isolating conflicting ones. Through these, each transformer block contains a shared expert across all tasks, where the common and task-specific denoising paths enable the diffusion model to construct its beneficial way of synergizing denoising tasks. Extensive experiments validate the effectiveness of our approach in improving both image quality and convergence rate, and further analysis demonstrates that Switch-DiT constructs tailored denoising paths across various generation scenarios., Comment: Project Page: https://byeongjun-park.github.io/Switch-DiT/

We study the twist-3 spin-orbit correlations of quarks described by the nucleon matrix elements of the parity-odd rank-2 tensor QCD operator (the parity-odd partner of the QCD energy-momentum tensor). Our treatment is based on the effective dynamics emerging from the spontaneous breaking of chiral symmetry and the mean-field picture of the nucleon in the large-$N_c$ limit. The twist-3 QCD operators are converted to effective operators, in which the QCD interactions are replaced by spin-flavor-dependent chiral interactions of the quarks with the pion field. We compute the nucleon matrix elements of the twist-3 effective operators and discuss the role of the chiral interactions in the spin-orbit correlations. We derive the first-quantized representation in the mean-field picture and develop a quantum-mechanical interpretation. The chiral interactions give rise to new spin-orbit couplings and qualitatively change the correlations compared to the quark model picture. We also derive the twist-3 matrix elements in the topological soliton picture where the quarks are integrated out (skyrmion). The methods used here can be extended to other QCD operators describing higher-twist nucleon structure and generalized parton distributions., Comment: 21 pages, 4 figures

We study the pole-skipping phenomenon within holographic axion theories, a common framework for studying strongly coupled systems with chemical potential ($\mu$) and momentum relaxation ($\beta$). Considering the backreaction characterized by $\mu$ and $\beta$, we encounter coupled equations of motion for the metric, gauge, and axion field, which are classified into spin-0, spin-1, and spin-2 channels. Employing gauge-invariant variables, we systematically address these equations and explore pole-skipping points within each sector using the near-horizon method. Our analysis reveals two classes of pole-skipping points: regular and singular pole-skipping points in which the latter is identified when standard linear differential equations exhibit singularity. Notably, pole-skipping points in the lower-half plane are regular, while those elsewhere are singular. This suggests that the pole-skipping point in the spin-0 channel, associated with quantum chaos, corresponds to a singular pole-skipping point. Additionally, we observe that the pole-skipping momentum, if purely real or imaginary for $\mu=\beta=0$, retains this characteristic for $\mu \neq0$ and $\beta \neq 0$., Comment: 30 pages, 3 figures

With the increasing complexity of generative AI models, post-training quantization (PTQ) has emerged as a promising solution for deploying hyper-scale models on edge devices such as mobile devices and TVs. Existing PTQ schemes, however, consume considerable time and resources, which could be a bottleneck in real situations where frequent model updates and multiple hyper-parameter tunings are required. As a cost-effective alternative, one-shot PTQ schemes have been proposed. Still, the performance is somewhat limited because they cannot consider the inter-layer dependency within the attention module, which is a very important feature of Transformers. In this paper, we thus propose a novel PTQ algorithm that balances accuracy and efficiency. The key idea of the proposed algorithm called aespa is to perform quantization layer-wise for efficiency while considering cross-layer dependency to preserve the attention score. Through extensive experiments on various language models and complexity analysis, we demonstrate that aespa is accurate and efficient in quantizing Transformer models., Comment: 17 pages, under review

Text generation is a compelling sub-field of natural language processing, aiming to generate human-readable text from input words. In particular, the decoder-only generative models, such as generative pre-trained transformer (GPT), are widely used for text generation, with two major computational stages: summarization and generation. Unlike the summarization stage, which can process the input tokens in parallel, the generation stage is difficult to accelerate due to its sequential generation of output tokens through iteration. Moreover, each iteration requires reading a whole model with little data reuse opportunity. Therefore, the workload of transformer-based text generation is severely memory-bound, making the external memory bandwidth system bottleneck. In this paper, we proposed a subarray-level processing-in-memory architecture named SAL-PIM, HBM-based PIM architecture for the end-to-end acceleration of transformer-based text generation. The SAL-PIM architecture includes three architectural features. First, the SAL-PIM architecture utilizes higher internal bandwidth by integrating multiple subarray-level arithmetic logic units with optimized data mapping schemes. Second, the SAL-PIM architecture adopts LUT-based linear interpolation to perform complex non-linear functions in PIM. Third, the SAL-PIM architecture accelerates end-to-end inference on PIM in text generation. Furthermore, to validate the SAL-PIM architecture, we built cycle-accurate simulator and implemented the SAL-PIM's logic units in 28-nm CMOS technology. As a result, when the input size is from 32 to 128 and the output size is from 1 to 256, SAL-PIM achieves a maximum of 4.72 times speedup and an average of 1.83 times speedup for the text generation based on the GPT-2 medium model compared to the server-level GPU., Comment: 14 pages, 15 figures

A partial flag variety is a smooth projective homogeneous variety admitting an action of a maximal torus $T$. Schubert varieties are $T$-invariant subvarieties of the partial flag varieties. We study toric Schubert varieties in Grassmannian varieties with respect to the action of the torus $T$. Indeed, we present an explicit description of the fan of a Gorenstein toric Schubert variety in a Grassmannian, and we prove that any Gorenstein toric Schubert variety in a Grassmannian variety is Fano.

We employ a deep learning method to deduce the \textit{bulk} spacetime from \textit{boundary} optical conductivity. We apply the neural ordinary differential equation technique, tailored for continuous functions such as the metric, to the typical class of holographic condensed matter models featuring broken translations: linear-axion models. We successfully extract the bulk metric from the boundary holographic optical conductivity. Furthermore, as an example for real material, we use experimental optical conductivity of $\text{UPd}_2\text{Al}_3$, a representative of heavy fermion metals in strongly correlated electron systems, and construct the corresponding bulk metric. To our knowledge, our work is the first illustration of deep learning bulk spacetime from \textit{boundary} holographic or experimental conductivity data., Comment: 30 pages, 8 figures

Recent progress in single-image 3D generation highlights the importance of multi-view coherency, leveraging 3D priors from large-scale diffusion models pretrained on Internet-scale images. However, the aspect of novel-view diversity remains underexplored within the research landscape due to the ambiguity in converting a 2D image into 3D content, where numerous potential shapes can emerge. Here, we aim to address this research gap by simultaneously addressing both consistency and diversity. Yet, striking a balance between these two aspects poses a considerable challenge due to their inherent trade-offs. This work introduces HarmonyView, a simple yet effective diffusion sampling technique adept at decomposing two intricate aspects in single-image 3D generation: consistency and diversity. This approach paves the way for a more nuanced exploration of the two critical dimensions within the sampling process. Moreover, we propose a new evaluation metric based on CLIP image and text encoders to comprehensively assess the diversity of the generated views, which closely aligns with human evaluators' judgments. In experiments, HarmonyView achieves a harmonious balance, demonstrating a win-win scenario in both consistency and diversity., Comment: Project page: https://byeongjun-park.github.io/HarmonyView/

One of the most well-known extragalactic sources in the sky, quasar 3C 454.3, shows a curved parsec-scale jet that has been exhaustively monitored with very-long-baseline interferometry (VLBI) over the recent years. In this work, we present a comprehensive analysis of four years of high-frequency VLBI observations at 43 GHz and 86 GHz, between 2013-2017, in total intensity and linear polarization. The images obtained from these observations enabled us to study the jet structure and the magnetic field topology of the source on spatial scales down to 4.6 parsec in projected distance. The kinematic analysis reveals the abrupt vanishing of at least four new superluminal jet features in a characteristic jet region (i.e., region C), which is located at an approximate distance of 0.6 milliarcseconds from the VLBI core. Our results support a model in which the jet bends, directing the relativistic plasma flow almost perfectly toward our line of sight, co-spatially with the region where components appear to stop., Comment: 15 pages, 7 figures

It is known from archival TESS data that the semi-detached Algol system XZ UMa is one of the candidate binary stars exhibiting short-period oscillations. We secured new high-resolution spectroscopic observations for the program target to better understand its binary and pulsation properties. From the echelle spectra, the radial velocities (RVs) of the eclipsing pair were derived, and the atmosphere parameters of the primary component were measured to be $v_{\rm A}\sin$$i = 80\pm7$ km s$^{-1}$, $T_{\rm eff,A}$ = $7940\pm120$ K, and [M/H] = $-0.15\pm0.20$. The combined solution of our double-lined RVs and the TESS data provides robust physical parameters for XZ UMa with mass and radius measurement precision of better than 2 \%. The outside-eclipse residuals from a mean light curve in the 0.002 phase bin were used for multifrequency analyses, and we extracted 32 significant frequencies (22 in $<$ 5.0 day$^{-1}$ and 10 in 39$-$52 day$^{-1}$). The low frequencies may be mostly aliasing sidelobes, while six of the high frequencies may be pulsation signals arising from the detached primary located inside the $\delta$ Sct domain. Their periods, pulsation constants, and pulsational-orbital-period ratios indicate that the mass-accretion primary star is a $\delta$ Sct pulsator and, hence, XZ UMa is an oscillating eclipsing Algol., Comment: 25 pages, including 7 figures and 5 tables, accepted for publication in PASJ

We present total intensity and linear polarization images of OJ287 at 1.68GHz, obtained through space-based VLBI observations with RadioAstron on April 16, 2016. The observations were conducted using a ground array consisting of the VLBA and the EVN. Ground-space fringes were detected with a maximum projected baseline length of 5.6 Earth's diameter, resulting in an angular resolution of 530 uas. With this unprecedented resolution at such a low frequency, the progressively bending jet structure of OJ287 has been resolved up to 10 pc of the projected distance from the radio core. In comparison with close-in-time VLBI observations at 15, 43, 86 GHz from MOJAVE and VLBA-BU-BLAZAR monitoring projects, we obtain the spectral index map showing the opaque core and optically thin jet components. The optically thick core has a brightness temperature of 10$^{13}$ K, and is further resolved into two sub-components at higher frequencies labeled C1 and C2. These sub-components exhibit a transition from optically thick to thin, with a SSA turnover frequency estimated to be 33 and 11.5 GHz, and a turnover flux density 4 and 0.7 Jy, respectively. Assuming a Doppler boosting factor of 10, the SSA values provide the estimate of the magnetic field strengths from SSA of 3.4 G for C1 and 1.0 G for C2. The magnetic field strengths assuming equipartition arguments are also estimated as 2.6 G and 1.6 G, respectively. The integrated degree of linear polarization is found to be approximately 2.5 %, with the electric vector position angle being well aligned with the local jet direction at the core region. This alignment suggests a predominant toroidal magnetic field, which is in agreement with the jet formation model that requires a helical magnetic field anchored to either the black hole ergosphere or the accretion disk. Further downstream, the jet seems to be predominantly threaded by a poloidal magnetic field., Comment: 15 pages, 11 figures, 3 tables

Brain-to-speech technology represents a fusion of interdisciplinary applications encompassing fields of artificial intelligence, brain-computer interfaces, and speech synthesis. Neural representation learning based intention decoding and speech synthesis directly connects the neural activity to the means of human linguistic communication, which may greatly enhance the naturalness of communication. With the current discoveries on representation learning and the development of the speech synthesis technologies, direct translation of brain signals into speech has shown great promise. Especially, the processed input features and neural speech embeddings which are given to the neural network play a significant role in the overall performance when using deep generative models for speech generation from brain signals. In this paper, we introduce the current brain-to-speech technology with the possibility of speech synthesis from brain signals, which may ultimately facilitate innovation in non-verbal communication. Also, we perform comprehensive analysis on the neural features and neural speech embeddings underlying the neurophysiological activation while performing speech, which may play a significant role in the speech synthesis works., Comment: 4 pages