Search

Recent large language models (LLMs) have enabled the development of advanced agentic systems that can integrate various tools and APIs to fulfill user queries through function calling. However, the deployment of these LLMs on the edge has not been explored since they typically require cloud-based infrastructure due to their substantial model size and computational demands. To this end, we present TinyAgent, an end-to-end framework for training and deploying task-specific small language model agents capable of function calling for driving agentic systems at the edge. We first show how to enable accurate function calling for open-source models via the LLMCompiler framework. We then systematically curate a high-quality dataset for function calling, which we use to fine-tune two small language models, TinyAgent-1.1B and 7B. For efficient inference, we introduce a novel tool retrieval method to reduce the input prompt length and utilize quantization to further accelerate the inference speed. As a driving application, we demonstrate a local Siri-like system for Apple's MacBook that can execute user commands through text or voice input. Our results show that our models can achieve, and even surpass, the function-calling capabilities of larger models like GPT-4-Turbo, while being fully deployed at the edge. We open-source our dataset, models, and installable package and provide a demo video for our MacBook assistant agent.

Pretrained large language models (LLMs) are currently state-of-the-art for solving the vast majority of natural language processing tasks. While many real-world applications still require fine-tuning to reach satisfactory levels of performance, many of them are in the low-data regime, making fine-tuning challenging. To address this, we propose LLM2LLM, a targeted and iterative data augmentation strategy that uses a teacher LLM to enhance a small seed dataset by augmenting additional data that can be used for fine-tuning on a specific task. LLM2LLM (1) fine-tunes a baseline student LLM on the initial seed data, (2) evaluates and extracts data points that the model gets wrong, and (3) uses a teacher LLM to generate synthetic data based on these incorrect data points, which are then added back into the training data. This approach amplifies the signal from incorrectly predicted data points by the LLM during training and reintegrates them into the dataset to focus on more challenging examples for the LLM. Our results show that LLM2LLM significantly enhances the performance of LLMs in the low-data regime, outperforming both traditional fine-tuning and other data augmentation baselines. LLM2LLM reduces the dependence on labor-intensive data curation and paves the way for more scalable and performant LLM solutions, allowing us to tackle data-constrained domains and tasks. We achieve improvements up to 24.2% on the GSM8K dataset, 32.6% on CaseHOLD, 32.0% on SNIPS, 52.6% on TREC and 39.8% on SST-2 over regular fine-tuning in the low-data regime using a Llama-2-7B student model. Our code is available at https://github.com/SqueezeAILab/LLM2LLM ., Comment: ACL 2024

Properly spin-adapted coupled-cluster theory for general open-shell configurations remains an elusive goal in electronic structure theory. In this contribution we examine Lindgren's normal-ordered exponential ansatz using spin-free excitation operators, with the aid of automatic equation generation software. We present a size-extensive reformulation of the unlinked working equations, and analyse the performance of the method with single and double excitations for simple molecular systems in terms of accuracy and size consistency., Comment: 9 pages, 2 figures

Molecular orbital theory is powerful both as a conceptual tool for understanding chemical bonding, and as a theoretical framework for ab initio quantum chemistry. Despite its undoubted success, MO theory has well documented shortcomings, most notably that it fails to correctly describe diradical states and homolytic bond fission. In this contribution, we introduce a generalised MO theory that includes spin-coupled radical states. We show through archetypical examples that when bonds break, the electronic state transitions between a small number of valence configurations, characterised by occupation of both delocalised molecular orbitals and spin-coupled localised orbitals. Our theory provides a model for chemical bonding that is both chemically intuitive and qualitatively accurate when combined with ab initio theory. Although exploitation of our theory presents significant challenges for classical computing, the predictable structure of spin-coupled states is ideally suited to algorithms that exploit quantum computers. Our approach provides a systematic route to overcoming the initial state overlap problem and unlocking the potential of quantum computational chemistry., Comment: 11 pages, 5 figures

The reasoning capabilities of the recent LLMs enable them to execute external function calls to overcome their inherent limitations, such as knowledge cutoffs, poor arithmetic skills, or lack of access to private data. This development has allowed LLMs to select and coordinate multiple functions based on the context to tackle more complex problems. However, current methods for function calling often require sequential reasoning and acting for each function which can result in high latency, cost, and sometimes inaccurate behavior. To address this, we introduce LLMCompiler, which executes functions in parallel to efficiently orchestrate multiple function calls. Drawing inspiration from the principles of classical compilers, LLMCompiler enables parallel function calling with three components: (i) a Function Calling Planner, formulating execution plans for function calling; (ii) a Task Fetching Unit, dispatching function calling tasks; and (iii) an Executor, executing these tasks in parallel. LLMCompiler automatically generates an optimized orchestration for the function calls and can be used with both open-source and closed-source models. We have benchmarked LLMCompiler on a range of tasks with different patterns of function calling. We observe consistent latency speedup of up to 3.7x, cost savings of up to 6.7x, and accuracy improvement of up to ~9% compared to ReAct. Our code is available at https://github.com/SqueezeAILab/LLMCompiler., Comment: ICML 2024

The "pretrain-then-finetune" paradigm is commonly adopted in the deployment of large language models. Low-Rank Adaptation (LoRA), a parameter-efficient fine-tuning method, is often employed to adapt a base model to a multitude of tasks, resulting in a substantial collection of LoRA adapters derived from one base model. We observe that this paradigm presents significant opportunities for batched inference during serving. To capitalize on these opportunities, we present S-LoRA, a system designed for the scalable serving of many LoRA adapters. S-LoRA stores all adapters in the main memory and fetches the adapters used by the currently running queries to the GPU memory. To efficiently use the GPU memory and reduce fragmentation, S-LoRA proposes Unified Paging. Unified Paging uses a unified memory pool to manage dynamic adapter weights with different ranks and KV cache tensors with varying sequence lengths. Additionally, S-LoRA employs a novel tensor parallelism strategy and highly optimized custom CUDA kernels for heterogeneous batching of LoRA computation. Collectively, these features enable S-LoRA to serve thousands of LoRA adapters on a single GPU or across multiple GPUs with a small overhead. Compared to state-of-the-art libraries such as HuggingFace PEFT and vLLM (with naive support of LoRA serving), S-LoRA can improve the throughput by up to 4 times and increase the number of served adapters by several orders of magnitude. As a result, S-LoRA enables scalable serving of many task-specific fine-tuned models and offers the potential for large-scale customized fine-tuning services. The code is available at https://github.com/S-LoRA/S-LoRA

Data standardization has become one of the leading methods neuroimaging researchers rely on for data sharing and reproducibility. Data standardization promotes a common framework through which researchers can utilize others' data. Yet, as of today, formatting datasets that adhere to community best practices requires technical expertise involving coding and considerable knowledge of file formats and standards. We describe ezBIDS, a tool for converting neuroimaging data and associated metadata to the Brain Imaging Data Structure (BIDS) standard. ezBIDS provides four unique features: (1) No installation or programming requirements. (2) Handling of both imaging and task events data and metadata. (3) Automated inference and guidance for adherence to BIDS. (4) Multiple data management options: download BIDS data to local system, or transfer to OpenNeuro.org or brainlife.io. In sum, ezBIDS requires neither coding proficiency nor knowledge of BIDS and is the first BIDS tool to offer guided standardization, support for task events conversion, and interoperability with OpenNeuro and brainlife.io.

We investigate the possibility of using a transcorrelated Hamiltonian to describe electron correlation. Amethod to obtain transcorrelatedwavefunctionswas developed based on the mathematical framework of the bi-variational principle. This involves the construction of an effective transcorrelated Hamiltonian matrix which can be solved in a self-consistent manner. This was optimised using a method we call Second Order Moment (SOM) minimisation to give highly accurate energies for some closed-shell atoms and helium-like ions. The effect of certain correlator terms on the description of electron-electron and electron-nuclear cusps were also examined graphically and some transcorrelated wavefunctions were compared against near-exact Hylleraas wavefunctions.

We present Suzaku observations of the Abell 98 (A98) triple galaxy cluster system and the purported intercluster filament. The three subclusters are expected to lie along a large scale cosmic filament. With partial azimuthal coverage of the northernmost cluster, we find that the inferred entropy profile of this relatively low mass cluster ($kT \approx 2.8$ keV) adheres to expectations from models of self-similar pure gravitational collapse in the region of the virial radius. There is evidence of extended structure beyond $r_{200}$ to the north of the northernmost cluster, along the merger axis, with properties consistent with what is expected for the warm-hot intergalacitc medium (WHIM; $kT = 0.11_{-0.02}^{+0.01}$ keV, $n_e = {7.6 \times 10^{-5}}^{+3.6 \times 10^{-5}}_{-3.6 \times 10^{-5}}$~cm$^{-3}$). No such emission is detected at the same radius in regions away from the merger axis, consistent with the expectation that the merger axis of this triple system lies along a large scale cosmic filament. In the bridge region between A98N and A98S, there is evidence of filamentary emission at the $2.2\sigma$ level, as well as a tentative detection of cool gas ($kT \sim 1$ keV). The entropy profile of this intercluster filament suggests that the A98 system is most likely aligned closer to the plane of the sky rather than along the line of sight. The structure to the north of the system, as well as in between A98N and A98S is indicative that the clusters are connected to a larger-scale structure spanning at least 4 Mpc., Comment: Submitted to ApJ. Comments are welcome

The recently proposed Conformer model has become the de facto backbone model for various downstream speech tasks based on its hybrid attention-convolution architecture that captures both local and global features. However, through a series of systematic studies, we find that the Conformer architecture's design choices are not optimal. After re-examining the design choices for both the macro and micro-architecture of Conformer, we propose Squeezeformer which consistently outperforms the state-of-the-art ASR models under the same training schemes. In particular, for the macro-architecture, Squeezeformer incorporates (i) the Temporal U-Net structure which reduces the cost of the multi-head attention modules on long sequences, and (ii) a simpler block structure of multi-head attention or convolution modules followed up by feed-forward module instead of the Macaron structure proposed in Conformer. Furthermore, for the micro-architecture, Squeezeformer (i) simplifies the activations in the convolutional block, (ii) removes redundant Layer Normalization operations, and (iii) incorporates an efficient depthwise down-sampling layer to efficiently sub-sample the input signal. Squeezeformer achieves state-of-the-art results of 7.5%, 6.5%, and 6.0% word-error-rate (WER) on LibriSpeech test-other without external language models, which are 3.1%, 1.4%, and 0.6% better than Conformer-CTC with the same number of FLOPs. Our code is open-sourced and available online., Comment: NeurIPS 2022

End-to-end neural network models achieve improved performance on various automatic speech recognition (ASR) tasks. However, these models perform poorly on edge hardware due to large memory and computation requirements. While quantizing model weights and/or activations to low-precision can be a promising solution, previous research on quantizing ASR models is limited. In particular, the previous approaches use floating-point arithmetic during inference and thus they cannot fully exploit efficient integer processing units. Moreover, they require training and/or validation data during quantization, which may not be available due to security or privacy concerns. To address these limitations, we propose an integer-only, zero-shot quantization scheme for ASR models. In particular, we generate synthetic data whose runtime statistics resemble the real data, and we use it to calibrate models during quantization. We apply our method to quantize QuartzNet, Jasper, and Conformer and show negligible WER degradation as compared to the full-precision baseline models, even without using any data. Moreover, we achieve up to 2.35x speedup on a T4 GPU and 4x compression rate, with a modest WER degradation of <1% with INT8 quantization.

Current Non-Orthogonal Configuration Interaction (NOCI) methods often use a set of Self-Consistent Field (SCF) states selected based on chemical intuition. However, it may be challenging to track these SCF states across a dissociation profile and the NOCI states recovered may be spin contaminated. In this paper, we propose a method of applying spin rotation on symmetry broken UHF (sb-UHF) states to generate a basis for NOCI. The dissociation of ethene was examined by localising spin rotation on each resulting carbene fragment. We show that this gives a size-consistent description of its dissociation and results in spin-pure states at all geometries. The dissociation was also studied with different orbitals, namely canonical UHF and Absolutely Localised Molecular Orbitals (ALMO). Furthermore, we demonstrate that the method can be used to restore spin symmetry of symmetry broken SCF wavefunctions for molecules of various sizes, marking an improvement over existing NOCI methods.

Recent literature suggests that there are two modes through which galaxies grow their stellar mass - a normal mode characterized by quasi-steady star formation, and a highly efficient starburst mode possibly triggered by stochastic events such as galaxy mergers. While these differences are established for extreme cases, the population of galaxies in-between these two regimes is poorly studied and it is not clear where the transition between these two modes of star formation occurs. We utilize ALMA observations of the CO J=3-2 line luminosity in a sample of 20 infrared luminous galaxies that lie in the intermediate range between normal and starburst galaxies at z ~ 0.25-0.6 in the COSMOS field to examine the gas content and star formation efficiency of these galaxies. We compare these quantities to the galaxies' deviation from the well-studied "main sequence" correlation between star formation rate and stellar mass (MS) and find that at log($SFR/SFR_{MS}$) < 0.6, a galaxy's distance to the main sequence is mostly driven by increased gas content, and not a more efficient star formation process., Comment: 21 pages, 10 figures

We present arc-second-resolution data in the radio, IR, optical and X-ray for 4C+19.44 (=PKS 1354+195), the longest and straightest quasar jet with deep X-ray observations. We report results from radio images with half to one arc-second angular resolution at three frequencies, plus HST and Spitzer data. The Chandra data allow us to measure the X-ray spectral index in 10 distinct regions along the 18 arcsec jet and compare with the radio index. The radio and X-ray spectral indices of the jet regions are consistent with a value of $\alpha =0.80$ throughout the jet, to within 2 sigma uncertainties. The X-ray jet structure to the south extends beyond the prominent radio jet and connects to the southern radio lobe, and there is extended X-ray emission in the direction of the unseen counter jet and coincident with the northern radio lobe. This jet is remarkable since its straight appearance over a large distance allows the geometry factors to be taken as fixed along the jet. Using the model of inverse Compton scattering of the cosmic microwave background (iC/CMB) by relativistic electrons, we find that the magnetic field strengths and Doppler factors are relatively constant along the jet. If instead the X-rays are synchrotron emission, they must arise from a population of electrons distinct from the particles producing the radio synchrotron spectrum., Comment: Accepted for ApJ

At redshift z = 2, when the Universe was just three billion years old, half of the most massive galaxies were extremely compact and had already exhausted their fuel for star formation(1-4). It is believed that they were formed in intense nuclear starbursts and that they ultimately grew into the most massive local elliptical galaxies seen today, through mergers with minor companions(5,6), but validating this picture requires higher-resolution observations of their centres than is currently possible. Magnification from gravitational lensing offers an opportunity to resolve the inner regions of galaxies(7). Here we report an analysis of the stellar populations and kinematics of a lensed z = 2.1478 compact galaxy, which surprisingly turns out to be a fast-spinning, rotationally supported disk galaxy. Its stars must have formed in a disk, rather than in a merger-driven nuclear starburst(8). The galaxy was probably fed by streams of cold gas, which were able to penetrate the hot halo gas until they were cut off by shock heating from the dark matter halo(9). This result confirms previous indirect indications(10-13) that the first galaxies to cease star formation must have gone through major changes not just in their structure, but also in their kinematics, to evolve into present-day elliptical galaxies., Comment: Nature (June 22, 2017)

The so-called star-forming main sequence of galaxies is the apparent tight relationship between the star formation rate and stellar mass of a galaxy. Previous studies exclude galaxies which are not strictly 'star forming' from the main sequence, because they do not lie on the same tight relation. Using local galaxies in the Sloan Digital Sky Survey we have classified galaxies according to their emission line ratios, and studied their location on the star formation rate - stellar mass plane. We find that galaxies form a sequence from the `blue cloud' galaxies which are actively forming stars, through a combination of composite, Seyfert, and LINER (Low-ionization nuclear emission-line region) galaxies, ending as 'red-and-dead' galaxies. The sequence supports an evolutionary pathway for galaxies in which star formation quenching by active galactic nuclei (AGN) plays a key role., Comment: 5 pages, 2 figures. Accepted by MNRAS 2015 September 15. First published online November 4, 2015

The relationship between galaxy star formation rates (SFR) and stellar masses ($M_\ast$) is re-examined using a mass-selected sample of $\sim$62,000 star-forming galaxies at $z \le 1.3$ in the COSMOS 2-deg$^2$ field. Using new far-infrared photometry from $Herschel$-PACS and SPIRE and $Spitzer$-MIPS 24 $\mu$m, along with derived infrared luminosities from the NRK method based on galaxies' locations in the restframe color-color diagram $(NUV - r)$ vs. $(r - K)$, we are able to more accurately determine total SFRs for our complete sample. At all redshifts, the relationship between median $SFR$ and $M_\ast$ follows a power-law at low stellar masses, and flattens to nearly constant SFR at high stellar masses. We describe a new parameterization that provides the best fit to the main sequence and characterizes the low mass power-law slope, turnover mass, and overall scaling. The turnover in the main sequence occurs at a characteristic mass of about $M_{0} \sim 10^{10} M_{\odot}$ at all redshifts. The low mass power-law slope ranges from 0.9-1.3 and the overall scaling rises in SFR as a function of $(1+z)^{4.12 \pm 0.10}$. A broken power-law fit below and above the turnover mass gives relationships of $SFR \propto M_{*}^{0.88 \pm 0.06}$ below the turnover mass and $SFR \propto M_{*}^{0.27 \pm 0.04}$ above the turnover mass. Galaxies more massive than $M_\ast \gtrsim 10^{10}\ M_{\rm \odot}$ have on average, a much lower specific star formation rate (sSFR) than would be expected by simply extrapolating the traditional linear fit to the main sequence found for less massive galaxies., Comment: 16 pages, 7 figures. Accepted for publication in ApJ