Search

The ability to perform complex tasks from detailed instructions is a key to many remarkable achievements of our species. As humans, we are not only capable of performing a wide variety of tasks but also very complex ones that may entail hundreds or thousands of steps to complete. Large language models and their more recent multimodal counterparts that integrate textual and visual inputs have achieved unprecedented success in performing complex tasks. Yet, most existing benchmarks are largely confined to single-modality inputs (either text or vision), narrowing the scope of multimodal assessments, particularly for instruction-following in multimodal contexts. To bridge this gap, we introduce the instructed-Virtual VISual Decision Making (iWISDM) environment engineered to generate a limitless array of vision-language tasks of varying complexity. Using iWISDM, we compiled three distinct benchmarks of instruction following visual tasks across varying complexity levels and evaluated several newly developed multimodal models on these benchmarks. Our findings establish iWISDM as a robust benchmark for assessing the instructional adherence of both existing and emergent multimodal models and highlight a large gap between these models' ability to precisely follow instructions with that of humans.The code of iWISDM is available on GitHub at https://github.com/BashivanLab/iWISDM.

Laser-induced gauge fields in neutral atoms serve as a means of mimicking the effects of a magnetic field, providing researchers with a platform to explore behaviors analogous to those observed in condensed matter systems under real magnetic fields. Here, we propose a method to generate chiral motion in atomic excitations within the neutral atomic ground-state manifolds. This is achieved through the application of polychromatic driving fields coupled to the ground-Rydberg transition, along with unconventional Rydberg pumping. The scheme offers the advantage of arbitrary adjustment of the effective magnetic flux by setting the relative phases between different external laser fields. Additionally, the effective interaction strength between the atomic ground states can be maintained at 10 kHz, surpassing the capabilities of the previous approach utilizing Floquet modulation. Notably, the proposed method can be readily extended to implement a hexagonal neutral atom lattice, serving as the fundamental unit in realizing the Haldane model., Comment: Some typographical errors in the timeline of Fig.5(c), Eq.(14), and Eq.(A1) of the published version have been corrected

We present the discovery and early observations of the nearby Type II supernova (SN) 2024ggi in NGC 3621 at 6.64 +/- 0.3 Mpc. The SN was caught 5.8 (+1.9 -2.9) hours after its explosion by the ATLAS survey. Early-phase, high-cadence, and multi-band photometric follow-up was performed by the Kinder (Kilonova Finder) project, collecting over 1000 photometric data points within a week. The combined o- and r-band light curves show a rapid rise of 3.3 magnitudes in 13.7 hours, much faster than SN 2023ixf (another recent, nearby, and well-observed SN II). Between 13.8 and 18.8 hours after explosion SN 2024ggi became bluer, with u-g colour dropping from 0.53 to 0.15 mag. The rapid blueward evolution indicates a wind shock breakout (SBO) scenario. No hour-long brightening expected for the SBO from a bare stellar surface was detected during our observations. The classification spectrum, taken 17 hours after the SN explosion, shows flash features of high-ionization species such as Balmer lines, He I, C III, and N III. Detailed light curve modeling reveals critical insights into the properties of the circumstellar material (CSM). Our favoured model has an explosion energy of 2 x 10^51 erg, a mass-loss rate of 10^-3 solar_mass/yr (with an assumed 10 km/s wind), and a confined CSM radius of 6 x 10^14 cm. The corresponding CSM mass is 0.4 solar_mass. Comparisons with SN 2023ixf highlight that SN 2024ggi has a smaller CSM density, resulting in a faster rise and fainter UV flux. The extensive dataset and the involvement of citizen astronomers underscore that a collaborative network is essential for SBO searches, leading to more precise and comprehensive SN characterizations., Comment: 11 pages, 5 figures in manuscript, 6 pages in appendix, submitted to ApJL

As a deep learning model, Visual Mamba (VMamba) has a low computational complexity and a global receptive field, which has been successful applied to image classification and detection. To extend its applications, we apply VMamba to crowd counting and propose a novel VMambaCC (VMamba Crowd Counting) model. Naturally, VMambaCC inherits the merits of VMamba, or global modeling for images and low computational cost. Additionally, we design a Multi-head High-level Feature (MHF) attention mechanism for VMambaCC. MHF is a new attention mechanism that leverages high-level semantic features to augment low-level semantic features, thereby enhancing spatial feature representation with greater precision. Building upon MHF, we further present a High-level Semantic Supervised Feature Pyramid Network (HS2PFN) that progressively integrates and enhances high-level semantic information with low-level semantic information. Extensive experimental results on five public datasets validate the efficacy of our approach. For example, our method achieves a mean absolute error of 51.87 and a mean squared error of 81.3 on the ShangHaiTech\_PartA dataset. Our code is coming soon.

Quantum embedding with transformers is a novel and promising architecture for quantum machine learning to deliver exceptional capability on near-term devices or simulators. The research incorporated a vision transformer (ViT) to advance quantum significantly embedding ability and results for a single qubit classifier with around 3 percent in the median F1 score on the BirdCLEF-2021, a challenging high-dimensional dataset. The study showcases and analyzes empirical evidence that our transformer-based architecture is a highly versatile and practical approach to modern quantum machine learning problems.

We report 850 $\mu$m continuum polarization observations toward the filamentary high-mass star-forming region NGC 2264, taken as part of the B-fields In STar forming Regions Observations (BISTRO) large program on the James Clerk Maxwell Telescope (JCMT). These data reveal a well-structured non-uniform magnetic field in the NGC 2264C and 2264D regions with a prevailing orientation around 30 deg from north to east. Field strengths estimates and a virial analysis for the major clumps indicate that NGC 2264C is globally dominated by gravity while in 2264D magnetic, gravitational, and kinetic energies are roughly balanced. We present an analysis scheme that utilizes the locally resolved magnetic field structures, together with the locally measured gravitational vector field and the extracted filamentary network. From this, we infer statistical trends showing that this network consists of two main groups of filaments oriented approximately perpendicular to one another. Additionally, gravity shows one dominating converging direction that is roughly perpendicular to one of the filament orientations, which is suggestive of mass accretion along this direction. Beyond these statistical trends, we identify two types of filaments. The type-I filament is perpendicular to the magnetic field with local gravity transitioning from parallel to perpendicular to the magnetic field from the outside to the filament ridge. The type-II filament is parallel to the magnetic field and local gravity. We interpret these two types of filaments as originating from the competition between radial collapsing, driven by filament self-gravity, and the longitudinal collapsing, driven by the region's global gravity., Comment: Accepted for publication in the Astrophysical Journal. 43 pages, 32 figures, and 4 tables (including Appendix)

Crowd counting has gained significant popularity due to its practical applications. However, mainstream counting methods ignore precise individual localization and suffer from annotation noise because of counting from estimating density maps. Additionally, they also struggle with high-density images.To address these issues, we propose an end-to-end model called Fine-Grained Extraction Network (FGENet). Different from methods estimating density maps, FGENet directly learns the original coordinate points that represent the precise localization of individuals.This study designs a fusion module, named Fine-Grained Feature Pyramid(FGFP), that is used to fuse feature maps extracted by the backbone of FGENet. The fused features are then passed to both regression and classification heads, where the former provides predicted point coordinates for a given image, and the latter determines the confidence level for each predicted point being an individual. At the end, FGENet establishes correspondences between prediction points and ground truth points by employing the Hungarian algorithm. For training FGENet, we design a robust loss function, named Three-Task Combination (TTC), to mitigate the impact of annotation noise. Extensive experiments are conducted on four widely used crowd counting datasets. Experimental results demonstrate the effectiveness of FGENet. Notably, our method achieves a remarkable improvement of 3.14 points in Mean Absolute Error (MAE) on the ShanghaiTech Part A dataset, showcasing its superiority over the existing state-of-the-art methods. Even more impressively, FGENet surpasses previous benchmarks on the UCF\_CC\_50 dataset with an astounding enhancement of 30.16 points in MAE., Comment: Accepted by 30th International Conference on MultiMedia Modeling

The research explores the potential of quantum deep learning models to address challenging machine learning problems that classical deep learning models find difficult to tackle. We introduce a novel model architecture that combines classical convolutional layers with a quantum neural network, aiming to surpass state-of-the-art accuracy while maintaining a compact model size. The experiment is to classify high-dimensional audio data from the Bird-CLEF 2021 dataset. Our evaluation focuses on key metrics, including training duration, model accuracy, and total model size. This research demonstrates the promising potential of quantum machine learning in enhancing machine learning tasks and solving practical machine learning challenges available today.

Quantum state preparation is vital in quantum computing and information processing. The ability to accurately and reliably prepare specific quantum states is essential for various applications. One of the promising applications of quantum computers is quantum simulation. This requires preparing a quantum state representing the system we are trying to simulate. This research introduces a novel simulation algorithm, the multi-Split-Steps Quantum Walk (multi-SSQW), designed to learn and load complicated probability distributions using parameterized quantum circuits (PQC) with a variational solver on classical simulators. The multi-SSQW algorithm is a modified version of the split-steps quantum walk, enhanced to incorporate a multi-agent decision-making process, rendering it suitable for modeling financial markets. The study provides theoretical descriptions and empirical investigations of the multi-SSQW algorithm to demonstrate its promising capabilities in probability distribution simulation and financial market modeling. Harnessing the advantages of quantum computation, the multi-SSQW models complex financial distributions and scenarios with high accuracy, providing valuable insights and mechanisms for financial analysis and decision-making. The multi-SSQW's key benefits include its modeling flexibility, stable convergence, and instantaneous computation. These advantages underscore its rapid modeling and prediction potential in dynamic financial markets., Comment: 13 pages, 13 figures. arXiv admin note: text overlap with arXiv:2302.12500

We study domain-adaptive image synthesis, the problem of teaching pretrained image generative models a new style or concept from as few as one image to synthesize novel images, to better understand the compositional image synthesis. We present a framework that leverages a pretrained class-conditional generation model and visual prompt tuning. Specifically, we propose a novel source class distilled visual prompt that learns disentangled prompts of semantic (e.g., class) and domain (e.g., style) from a few images. Learned domain prompt is then used to synthesize images of any classes in the style of target domain. We conduct studies on various target domains with the number of images ranging from one to a few to many, and show qualitative results which show the compositional generalization of our method. Moreover, we show that our method can help improve zero-shot domain adaptation classification accuracy., Comment: tech report

Pre-trained large text-to-image models synthesize impressive images with an appropriate use of text prompts. However, ambiguities inherent in natural language and out-of-distribution effects make it hard to synthesize image styles, that leverage a specific design pattern, texture or material. In this paper, we introduce StyleDrop, a method that enables the synthesis of images that faithfully follow a specific style using a text-to-image model. The proposed method is extremely versatile and captures nuances and details of a user-provided style, such as color schemes, shading, design patterns, and local and global effects. It efficiently learns a new style by fine-tuning very few trainable parameters (less than $1\%$ of total model parameters) and improving the quality via iterative training with either human or automated feedback. Better yet, StyleDrop is able to deliver impressive results even when the user supplies only a single image that specifies the desired style. An extensive study shows that, for the task of style tuning text-to-image models, StyleDrop implemented on Muse convincingly outperforms other methods, including DreamBooth and textual inversion on Imagen or Stable Diffusion. More results are available at our project website: https://styledrop.github.io, Comment: Preprint. Project page at https://styledrop.github.io

Since acquiring perfect supervision is usually difficult, real-world machine learning tasks often confront inaccurate, incomplete, or inexact supervision, collectively referred to as weak supervision. In this work, we present WSAUC, a unified framework for weakly supervised AUC optimization problems, which covers noisy label learning, positive-unlabeled learning, multi-instance learning, and semi-supervised learning scenarios. Within the WSAUC framework, we first frame the AUC optimization problems in various weakly supervised scenarios as a common formulation of minimizing the AUC risk on contaminated sets, and demonstrate that the empirical risk minimization problems are consistent with the true AUC. Then, we introduce a new type of partial AUC, specifically, the reversed partial AUC (rpAUC), which serves as a robust training objective for AUC maximization in the presence of contaminated labels. WSAUC offers a universal solution for AUC optimization in various weakly supervised scenarios by maximizing the empirical rpAUC. Theoretical and experimental results under multiple settings support the effectiveness of WSAUC on a range of weakly supervised AUC optimization tasks., Comment: Accepted by IEEE TPAMI

We present observations of polarized dust emission at 850 $\mu$m from the L43 molecular cloud which sits in the Ophiuchus cloud complex. The data were taken using SCUBA-2/POL-2 on the James Clerk Maxwell Telescope as a part of the BISTRO large program. L43 is a dense ($N_{\rm H_2}\sim 10^{22}$-10$^{23}$ cm$^{-2}$) complex molecular cloud with a submillimetre-bright starless core and two protostellar sources. There appears to be an evolutionary gradient along the isolated filament that L43 is embedded within, with the most evolved source closest to the Sco OB2 association. One of the protostars drives a CO outflow that has created a cavity to the southeast. We see a magnetic field that appears to be aligned with the cavity walls of the outflow, suggesting interaction with the outflow. We also find a magnetic field strength of up to $\sim$160$\pm$30 $\mu$G in the main starless core and up to $\sim$90$\pm$40 $\mu$G in the more diffuse, extended region. These field strengths give magnetically super- and sub-critical values respectively and both are found to be roughly trans-Alfv\'enic. We also present a new method of data reduction for these denser but fainter objects like starless cores., Comment: Accepted for publication in ApJ. 23 pages, 9 figures (7 main text, 2 appendix)

A first hydrostatic core (FHC) is proposed to form after the initial collapse of a prestellar core, as a seed of a Class 0 protostar. FHCs are difficult to observe because they are small, compact, embedded, and short lived. In this work, we explored the physical properties of two well-known FHC candidates, B1-bN and B1-bS, by comparing interferometric data from Submillimeter Array (SMA) 1.1 and 1.3 mm and Atacama Large Millimeter/submillimeter Array (ALMA) 870 $\mu$m observations with simulated synthesis images of the two sources. The simulated images are based on a simple model containing a single, hot compact first-core-like component at the center surrounded by a large-scale, cold and dusty envelope described by a broken power-law density distribution with an index, $\alpha$. Our results show that the hot compact components of B1-bN and B1-bS can be described by temperatures of \sim 500 K with a size of \sim 4 au, which are in agreement with theoretical predictions of an FHC. If the $\alpha$ inside the broken radii is fixed to -1.5, we find $\alpha$ \sim -2.9 and \sim -3.3 outside the broken radii for B1-bN and B1-bS, respectively, consistent with theoretical calculations of a collapsing, bounded envelope and previous observations. Comparing the density and temperature profiles of the two sources with radiation-hydrodynamic simulations of an FHC, we find both sources lie close to, but before, the second collapse stage. We suggest that B1-bS may have started the collapsing process earlier compared to B1-bN, since a larger discontinuity point is found in its density profile.

Quantum computing holds great potential for advancing the limitations of machine learning algorithms to handle higher dimensions of data and reduce overall training parameters in deep learning (DL) models. This study uses a trainable variational quantum circuit (VQC) on a gate-based quantum computing model to investigate the potential for quantum benefit in a model-free reinforcement learning problem. Through a comprehensive investigation and evaluation of the current model and capabilities of quantum computers, we designed and trained a novel hybrid quantum neural network based on the latest Qiskit and PyTorch framework. We compared its performance with a full-classical CNN with and without an incorporated VQC. Our research provides insights into the potential of deep quantum learning to solve a maze problem and, potentially, other reinforcement learning problems. We conclude that reinforcement learning problems can be practical with reasonable training epochs. Moreover, a comparative study of full-classical and hybrid quantum neural networks is discussed to understand these two approaches' performance, advantages, and disadvantages to deep-Q learning problems, especially on larger-scale maze problems larger than 4x4.

We construct the four-derivative supersymmetric extension of $(1,0), 6D$ supergravity coupled to Yang-Mills and hypermultiplets. The hypermultiplet scalars are taken to parametrize the quaternionic projective space $Hp(n)=Sp(n,1)/Sp(n)\times Sp(1)_R$. The hyperscalar kinetic term is not deformed, and the quaternionic K\"ahler structure and symmetries of $Hp(n)$ are preserved. The result is a three parameter Lagrangian supersymmetric up to first order in these parameters. Considering the case of $Hp(1)$ we compare our result with that obtained from the compactification of $10D$ heterotic supergravity on four-torus, consistently truncated to $N=(1,0)$, in which the hyperscalars parametrize $SO(4,1)/SO(4)$. We find that depending on how $Sp(1) \subset Sp(1,1)$ is embedded in $SO(4)$, the results agree for a specific value of the parameter that governs the higher derivative hypermultiplet couplings., Comment: 29 pages

In recent years, there has been an emerging trend of combining two innovations in computer science and physics to achieve better computation capability. Exploring the potential of quantum computation to achieve highly efficient performance in various tasks is a vital development in engineering and a valuable question in sciences, as it has a significant potential to provide exponential speedups for technologically complex problems that are specifically advantageous to quantum computers. However, one key issue in unleashing this potential is constructing an efficient approach to load classical data into quantum states that can be executed by quantum computers or quantum simulators on classical hardware. Therefore, the split-step quantum walks (SSQW) algorithm was proposed to address this limitation. We facilitate SSQW to design parameterized quantum circuits (PQC) that can generate probability distributions and optimize the parameters to achieve the desired distribution using a variational solver. A practical example of implementing SSQW using Qiskit has been released as open-source software. Showing its potential as a promising method for generating desired probability amplitude distributions highlights the potential application of SSQW in option pricing through quantum simulation., Comment: 11 pages, 7 figures

We present BISTRO Survey 850 {\mu}m dust emission polarisation observations of the L1495A-B10 region of the Taurus molecular cloud, taken at the JCMT. We observe a roughly triangular network of dense filaments. We detect 9 of the dense starless cores embedded within these filaments in polarisation, finding that the plane-of-sky orientation of the core-scale magnetic field lies roughly perpendicular to the filaments in almost all cases. We also find that the large-scale magnetic field orientation measured by Planck is not correlated with any of the core or filament structures, except in the case of the lowest-density core. We propose a scenario for early prestellar evolution that is both an extension to, and consistent with, previous models, introducing an additional evolutionary transitional stage between field-dominated and matter-dominated evolution, observed here for the first time. In this scenario, the cloud collapses first to a sheet-like structure. Uniquely, we appear to be seeing this sheet almost face-on. The sheet fragments into filaments, which in turn form cores. However, the material must reach a certain critical density before the evolution changes from being field-dominated to being matter-dominated. We measure the sheet surface density and the magnetic field strength at that transition for the first time and show consistency with an analytical prediction that had previously gone untested for over 50 years (Mestel 1965)., Comment: 14 pages, 5 figures. ApJ accepted

We study the HII regions associated with the NGC 6334 molecular cloud observed in the sub-millimeter and taken as part of the B-fields In STar-forming Region Observations (BISTRO) Survey. In particular, we investigate the polarization patterns and magnetic field morphologies associated with these HII regions. Through polarization pattern and pressure calculation analyses, several of these bubbles indicate that the gas and magnetic field lines have been pushed away from the bubble, toward an almost tangential (to the bubble) magnetic field morphology. In the densest part of NGC 6334, where the magnetic field morphology is similar to an hourglass, the polarization observations do not exhibit observable impact from HII regions. We detect two nested radial polarization patterns in a bubble to the south of NGC 6334 that correspond to the previously observed bipolar structure in this bubble. Finally, using the results of this study, we present steps (incorporating computer vision; circular Hough Transform) that can be used in future studies to identify bubbles that have physically impacted magnetic field lines., Comment: Accepted for publication in Astrophysical Journal (ApJ)

We introduce the MAsked Generative VIdeo Transformer, MAGVIT, to tackle various video synthesis tasks with a single model. We introduce a 3D tokenizer to quantize a video into spatial-temporal visual tokens and propose an embedding method for masked video token modeling to facilitate multi-task learning. We conduct extensive experiments to demonstrate the quality, efficiency, and flexibility of MAGVIT. Our experiments show that (i) MAGVIT performs favorably against state-of-the-art approaches and establishes the best-published FVD on three video generation benchmarks, including the challenging Kinetics-600. (ii) MAGVIT outperforms existing methods in inference time by two orders of magnitude against diffusion models and by 60x against autoregressive models. (iii) A single MAGVIT model supports ten diverse generation tasks and generalizes across videos from different visual domains. The source code and trained models will be released to the public at https://magvit.cs.cmu.edu., Comment: CVPR 2023 highlight

We present 850 $\mu$m dust polarization observations of the massive DR21 filament from the B-fields In STar-forming Region Observations (BISTRO) survey, using the POL-2 polarimeter and the SCUBA-2 camera on the James Clerk Maxwell Telescope. We detect ordered magnetic fields perpendicular to the parsec-scale ridge of the DR21 main filament. In the sub-filaments, the magnetic fields are mainly parallel to the filamentary structures and smoothly connect to the magnetic fields of the main filament. We compare the POL-2 and Planck dust polarization observations to study the magnetic field structures of the DR21 filament on 0.1--10 pc scales. The magnetic fields revealed in the Planck data are well aligned with those of the POL-2 data, indicating a smooth variation of magnetic fields from large to small scales. The plane-of-sky magnetic field strengths derived from angular dispersion functions of dust polarization are 0.6--1.0 mG in the DR21 filament and $\sim$ 0.1 mG in the surrounding ambient gas. The mass-to-flux ratios are found to be magnetically supercritical in the filament and slightly subcritical to nearly critical in the ambient gas. The alignment between column density structures and magnetic fields changes from random alignment in the low-density ambient gas probed by Planck to mostly perpendicular in the high-density main filament probed by JCMT. The magnetic field structures of the DR21 filament are in agreement with MHD simulations of a strongly magnetized medium, suggesting that magnetic fields play an important role in shaping the DR21 main filament and sub-filaments., Comment: 26 pages, 13 figures, ApJ accepted