Search

Zero-shot multi-label recognition (MLR) with Vision-Language Models (VLMs) faces significant challenges without training data, model tuning, or architectural modifications. Existing approaches require prompt tuning or architectural adaptations, limiting zero-shot applicability. Our work proposes a novel solution treating VLMs as black boxes, leveraging scores without training data or ground truth. Using large language model insights on object co-occurrence, we introduce compound prompts grounded in realistic object combinations. Analysis of these prompt scores reveals VLM biases and ``AND''/``OR'' signal ambiguities, notably that maximum compound scores are surprisingly suboptimal compared to second-highest scores. We address these through a debiasing and score-fusion algorithm that corrects image bias and clarifies VLM response behaviors. Our method enhances other zero-shot approaches, consistently improving their results. Experiments show superior mean Average Precision (mAP) compared to methods requiring training data, achieved through refined object ranking for robust zero-shot MLR.

This paper introduces a new mathematical framework for analysis and optimization of tensor expressions within an enclosing loop. Tensors are multi-dimensional arrays of values. They are common in high performance computing (HPC) and machine learning domains. Our framework extends Scalar Evolution - an important optimization pass implemented in both LLVM and GCC - to tensors. Scalar Evolution (SCEV) relies on the theory of `Chain of Recurrences' for its mathematical underpinnings. We use the same theory for Tensor Evolution (TeV). While some concepts from SCEV map easily to TeV -- e.g. element-wise operations; tensors introduce new operations such as concatenation, slicing, broadcast, reduction, and reshape which have no equivalent in scalars and SCEV. Not all computations are amenable to TeV analysis but it can play a part in the optimization and analysis parts of ML and HPC compilers. Also, for many mathematical/compiler ideas, applications may go beyond what was initially envisioned, once others build on it and take it further. We hope for a similar trajectory for the tensor-evolution concept.

There is significant concern about the impact of generative AI on society. Modern AI tools are capable of generating ever more realistic text, images, and videos, and functional code, from minimal prompts. Accompanying this rise in ability and usability, there is increasing alarm about the misuses to which these tools can be put, and the intentional and unintentional harms to individuals and society that may result. In this paper, we argue that \emph{agency} is the appropriate lens to study these harms and benefits, but that doing so will require advancement in the theory of agency, and advancement in how this theory is applied in (agent-based) models.

The growing interest in employing large language models (LLMs) for decision-making in social and economic contexts has raised questions about their potential to function as agents in these domains. A significant number of societal problems involve the distribution of resources, where fairness, along with economic efficiency, play a critical role in the desirability of outcomes. In this paper, we examine whether LLM responses adhere to fundamental fairness concepts such as equitability, envy-freeness, and Rawlsian maximin, and investigate their alignment with human preferences. We evaluate the performance of several LLMs, providing a comparative benchmark of their ability to reflect these measures. Our results demonstrate a lack of alignment between current LLM responses and human distributional preferences. Moreover, LLMs are unable to utilize money as a transferable resource to mitigate inequality. Nonetheless, we demonstrate a stark contrast when (some) LLMs are tasked with selecting from a predefined menu of options rather than generating one. In addition, we analyze the robustness of LLM responses to variations in semantic factors (e.g. intentions or personas) or non-semantic prompting changes (e.g. templates or orderings). Finally, we highlight potential strategies aimed at enhancing the alignment of LLM behavior with well-established fairness concepts.

Magnetic topological insulators (MTIs) host topologically protected edge states, but the role that these edge states play in electronic transport remains unclear. Using scanning superconducting quantum interference device (SQUID) microscopy, we performed local measurements of the current distribution in a quantum anomalous Hall (QAH) insulator at large bias currents, where the quantization of the conductivity tensor breaks down. We find that bulk currents in the channel interior coexist with edge currents at the sample boundary. While the position of the edge current changes with the reversal of the magnetic field, it does not depend on the current direction. To understand our observations, we introduce a model which includes contributions from both the sample magnetization and currents driven by chemical potential gradients. To parameterize our model, we use local measurements of the chemical potential induced changes in the sample magnetization. Our model reveals that the observed edge currents can be understood as changes in the magnetization generated by the electrochemical potential distribution in the sample under bias. Our work underscores the complexity of electronic transport in MTIs and highlights both the value and challenges of using magnetic imaging to disentangle various contributions to the electronic transport signatures.

With rapid advances in qubit technologies, techniques for localizing, modulating, and measuring RF fields and their impact on qubit performance are of the utmost importance. Here, we demonstrate that flux-channeling from a permalloy nanowire can be used to achieve localized spatial modulation of an RF field and that the modulated field can be mapped with high resolution using the Rabi oscillations of an NV center. Rabi maps reveal ~100 mm wavelength microwaves concentrated in sub-300 nm-scale regions with up to ~16$\times$ power enhancement. This modulation is robust over a 20 dBm power range and has no adverse impact on NV $T_2$ coherence time. Micromagnetic simulations confirm that the modulated field results from the nanowire's stray field through its constructive/destructive interference with the incident RF field. Our findings provide a new pathway for controlling qubits, amplifying RF signals, and mapping local fields in various on-chip RF technologies.

We report charge to spin conversion in a hybrid heterostructure comprised of atomically thin bismuth (Bi) confined between a silicon carbide (SiC) substrate and epitaxial graphene (EG). We confirm composition, dimensionality, and a 96.5 \% intercalation coverage using X-ray photolectron spectroscopy, scanning transmission microscopy, low energy electron diffraction, and Raman spectroscopy. Electrical transport measurements show signs of weak antilocalization in the heterostructure, consistent with spin-orbit coupling in this hybrid heterostructure. Spin torque ferromagnetic resonance measurements in permalloy/EG/2D-Bi heterostructures probe charge-to-spin conversion and revealing that an in plane polarization of the spin current, perpendicular to the charge current. The ratio of the in-plane to out-of-plane torque is 3.75 times higher than in hydrogenated graphene control samples.

Considering the difficulty of interpreting generative model output, there is significant current research focused on determining meaningful evaluation metrics. Several recent approaches utilize "precision" and "recall," borrowed from the classification domain, to individually quantify the output fidelity (realism) and output diversity (representation of the real data variation), respectively. With the increase in metric proposals, there is a need for a unifying perspective, allowing for easier comparison and clearer explanation of their benefits and drawbacks. To this end, we unify a class of kth-nearest-neighbors (kNN)-based metrics under an information-theoretic lens using approaches from kNN density estimation. Additionally, we propose a tri-dimensional metric composed of Precision Cross-Entropy (PCE), Recall Cross-Entropy (RCE), and Recall Entropy (RE), which separately measure fidelity and two distinct aspects of diversity, inter- and intra-class. Our domain-agnostic metric, derived from the information-theoretic concepts of entropy and cross-entropy, can be dissected for both sample- and mode-level analysis. Our detailed experimental results demonstrate the sensitivity of our metric components to their respective qualities and reveal undesirable behaviors of other metrics.

Embedding models are crucial for tasks in Information Retrieval (IR) and semantic similarity measurement, yet their handling of longer texts and associated positional biases remains underexplored. In this study, we investigate the impact of content position and input size on text embeddings. Our experiments reveal that embedding models, irrespective of their positional encoding mechanisms, disproportionately prioritize the beginning of an input. Ablation studies demonstrate that insertion of irrelevant text or removal at the start of a document reduces cosine similarity between altered and original embeddings by up to 12.3% more than ablations at the end. Regression analysis further confirms this bias, with sentence importance declining as position moves further from the start, even with with content-agnosticity. We hypothesize that this effect arises from pre-processing strategies and chosen positional encoding techniques. These findings quantify the sensitivity of retrieval systems and suggest a new lens towards embedding model robustness., Comment: 13 pages, 11 figures, NeurIPS

Channel and spatial attention mechanisms introduced by earlier works enhance the representation abilities of deep convolutional neural networks (CNNs) but often lead to increased parameter and computation costs. While recent approaches focus solely on efficient feature context modeling for channel attention, we aim to model both channel and spatial attention comprehensively with minimal parameters and reduced computation. Leveraging the principles of relational modeling in graphs, we introduce a constant-parameter module, STEAM: Squeeze and Transform Enhanced Attention Module, which integrates channel and spatial attention to enhance the representation power of CNNs. To our knowledge, we are the first to propose a graph-based approach for modeling both channel and spatial attention, utilizing concepts from multi-head graph transformers. Additionally, we introduce Output Guided Pooling (OGP), which efficiently captures spatial context to further enhance spatial attention. We extensively evaluate STEAM for large-scale image classification, object detection and instance segmentation on standard benchmark datasets. STEAM achieves a 2% increase in accuracy over the standard ResNet-50 model with only a meager increase in GFLOPs. Furthermore, STEAM outperforms leading modules ECA and GCT in terms of accuracy while achieving a three-fold reduction in GFLOPs.

Source separation is the process of isolating individual sounds in an auditory mixture of multiple sounds [1], and has a variety of applications ranging from speech enhancement and lyric transcription [2] to digital audio production for music. Furthermore, Automatic Music Transcription (AMT) is the process of converting raw music audio into sheet music that musicians can read [3]. Historically, these tasks have faced challenges such as significant audio noise, long training times, and lack of free-use data due to copyright restrictions. However, recent developments in deep learning have brought new promising approaches to building low-distortion stems and generating sheet music from audio signals [4]. Using spectrogram masking, deep neural networks, and the MuseScore API, we attempt to create an end-to-end pipeline that allows for an initial music audio mixture (e.g...wav file) to be separated into instrument stems, converted into MIDI files, and transcribed into sheet music for each component instrument.

As text-to-image models grow increasingly powerful and complex, their burgeoning size presents a significant obstacle to widespread adoption, especially on resource-constrained devices. This paper presents a pioneering study on post-training pruning of Stable Diffusion 2, addressing the critical need for model compression in text-to-image domain. Our study tackles the pruning techniques for the previously unexplored multi-modal generation models, and particularly examines the pruning impact on the textual component and the image generation component separately. We conduct a comprehensive comparison on pruning the model or the single component of the model in various sparsities. Our results yield previously undocumented findings. For example, contrary to established trends in language model pruning, we discover that simple magnitude pruning outperforms more advanced techniques in text-to-image context. Furthermore, our results show that Stable Diffusion 2 can be pruned to 38.5% sparsity with minimal quality loss, achieving a significant reduction in model size. We propose an optimal pruning configuration that prunes the text encoder to 47.5% and the diffusion generator to 35%. This configuration maintains image generation quality while substantially reducing computational requirements. In addition, our work uncovers intriguing questions about information encoding in text-to-image models: we observe that pruning beyond certain thresholds leads to sudden performance drops (unreadable images), suggesting that specific weights encode critical semantics information. This finding opens new avenues for future research in model compression, interoperability, and bias identification in text-to-image models. By providing crucial insights into the pruning behavior of text-to-image models, our study lays the groundwork for developing more efficient and accessible AI-driven image generation systems

Solar meridional circulation, which manifests as poleward flow near the surface, is a relatively weak flow. While meridional circulation has been measured through various local helioseismic techniques, there is a lack of consensus about the nature of the depth profile and location of return flow, owing to its small amplitude and poor signal-to-noise ratio in observations. The measurements are strongly hampered by systematic effects, whose amplitudes are comparable to the signal induced by the flow and modelling them is therefore crucial. The removal of the center-to-limb systematic, which is the largest known feature hampering the inference of meridional flow, has been heuristically performed in helioseismic analyses, but it's effect on global modes is not fully understood or modelled. Here, we propose both a way to model the center-to-limb systematic and a method for estimation of meridional flow using global helioseismic cross-spectral analysis. We demonstrate that the systematic cannot be ignored while modelling the mode-coupling cross-spectral measurement and thus is critical for the inference of meridional circulation. We also show that inclusion of a model for the center-to-limb systematic improves shallow meridional circulation estimates from cross-spectral analysis., Comment: 26 pages, 23 figures, accepted for publication in ApJ

The advancement of 5G and NextG networks through Open Radio Access Network (O-RAN) architecture enables a shift toward virtualized, modular, and disaggregated configurations. A core component of O-RAN is the RAN Intelligent Controller (RIC), which manages RAN using machine learning-driven xApps that access sensitive data from RAN and User Equipment (UE), stored in the near Real-Time RIC (Near-RT RIC) database. This shared, open environment increases the risk of unauthorized data exposure. To address these concerns, this paper proposes a zero-trust RIC (ZT-RIC) framework that preserves data privacy across the RIC platform, including the RIC database, xApps, and E2 interface. ZT-RIC employs Inner Product Functional Encryption (IPFE) to encrypt RAN/UE data at the base station, preventing leaks through the E2 interface and shared database. Additionally, ZT-RIC enables xApps to perform inference on encrypted data without exposing sensitive information. For evaluation, a state-of-the-art InterClass xApp, which detects jamming signals using RAN key performance metrics (KPMs), is implemented. Testing on an LTE/5G O-RAN testbed shows that ZT-RIC preserves data confidentiality while achieving 97.9% accuracy in jamming detection and meeting sub-second latency requirements, with a round-trip time (RTT) of 0.527 seconds., Comment: This paper has been accepted to CCNC 2025

Topological effects manifest in a variety of lattice geometries. While square lattices, due to their simplicity, have been used for models supporting nontrivial topology, several exotic topological phenomena such as Dirac points, Weyl points and Haldane phases are most commonly supported by non-square lattices. Examples of prototypical non-square lattices include the honeycomb lattice of graphene and the Kagome lattice, both of which break fundamental symmetries and can exhibit quantized transport, especially when long-range hoppings and gauge fields are incorporated. The challenge of controllably realizing long-range hoppings and gauge fields has motivated a large body of research focused on harnessing lattices encoded in "synthetic" dimensions. Photons in particular have many internal degrees of freedom and hence show promise for implementing these synthetic dimensions; however, most photonic synthetic dimensions has hitherto created 1D or 2D square lattices. Here we show that non-square lattice Hamiltonians can be implemented using Floquet synthetic dimensions. Our construction uses dynamically modulated ring resonators and provides the capacity for direct $k$-space engineering of lattice Hamiltonians. Such a construction lifts constraints on the orthogonality of lattice vectors that make square geometries simpler to implement, and instead transfers the complexity to the engineering of complex Floquet drive signals. We simulate topological signatures of the Haldane and the brick-wall Haldane model and observe them to be robust in the presence of external optical drive and photon loss, and discuss unique characteristics of their topological transport when implemented on these Floquet lattices. Our proposal demonstrates the potential of driven-dissipative Floquet synthetic dimensions as a new architecture for $k$-space Hamiltonian simulation of high-dimensional lattice geometries.

One Health issues, such as the spread of highly pathogenic avian influenza (HPAI), present significant challenges at the intersection of human, animal, and environmental health. Recent H5N1 outbreaks underscore the need for comprehensive modeling that capture the complex interactions between various entities in these interconnected ecosystems, encompassing livestock, wild birds, and human populations. To support such efforts, we present a synthetic spatiotemporal gridded dataset for the contiguous United States, referred to as a digital similar. The methodology for constructing this digital similar involves fusing diverse datasets using statistical and optimization techniques. The livestock component includes farm-level representations of multiple livestock types -- cattle, poultry, hogs, and sheep -- including further categorization into subtypes, such as milk and beef cows, chicken, turkeys, ducks, etc. It also includes location-level data for livestock-product processing centers. Weekly abundance data for key wild bird species involved in avian flu transmission are included along with temporal networks of movements. Gridded distributions of the human population, along with demographic and occupational features, capture the placement of agricultural workers and the general population. The digital similar is verified and validated in multiple ways.This dataset aims to provide a comprehensive basis for modeling complex phenomena at the wild-domestic-human interfaces.

Music source separation (MSS) is a task that involves isolating individual sound sources, or stems, from mixed audio signals. This paper presents an ensemble approach to MSS, combining several state-of-the-art architectures to achieve superior separation performance across traditional Vocal, Drum, and Bass (VDB) stems, as well as expanding into second-level hierarchical separation for sub-stems like kick, snare, lead vocals, and background vocals. Our method addresses the limitations of relying on a single model by utilising the complementary strengths of various models, leading to more balanced results across stems. For stem selection, we used the harmonic mean of Signal-to-Noise Ratio (SNR) and Signal-to-Distortion Ratio (SDR), ensuring that extreme values do not skew the results and that both metrics are weighted effectively. In addition to consistently high performance across the VDB stems, we also explored second-level hierarchical separation, revealing important insights into the complexities of MSS and how factors like genre and instrumentation can influence model performance. While the second-level separation results show room for improvement, the ability to isolate sub-stems marks a significant advancement. Our findings pave the way for further research in MSS, particularly in expanding model capabilities beyond VDB and improving niche stem separations such as guitar and piano.

Ballots are the core records of elections. Electronic records of actual ballots cast (cast vote records) are available to the public in some jurisdictions. However, they have been released in a variety of formats and have not been independently evaluated. Here we introduce a database of cast vote records from the 2020 U.S. general election. We downloaded publicly available unstandardized cast vote records, standardized them into a multi-state database, and extensively compared their totals to certified election results. Our release includes vote records for President, Governor, U.S. Senate and House, and state upper and lower chambers -- covering 42.7 million voters in 20 states who voted for more than 2,204 candidates. This database serves as a uniquely granular administrative dataset for studying voting behavior and election administration. Using this data, we show that in battleground states, 1.9 percent of solid Republicans (as defined by their congressional and state legislative voting) in our database split their ticket for Joe Biden, while 1.2 percent of solid Democrats split their ticket for Donald Trump., Comment: 26 pages and appendix

Understanding the internal mechanisms of GPT-style transformers, particularly their capacity to perform in-context learning (ICL), is critical for advancing AI alignment and interpretability. In-context learning allows transformers to generalize during inference without modifying their weights, yet the precise operations driving this capability remain largely opaque. This paper presents an investigation into the mechanistic interpretability of these transformers, focusing specifically on their ability to learn and predict affine recurrences as an ICL task. To address this, we trained a custom three-layer transformer to predict affine recurrences and analyzed the model's internal operations using both empirical and theoretical approaches. Our findings reveal that the model forms an initial estimate of the target sequence using a copying mechanism in the zeroth layer, which is subsequently refined through negative similarity heads in the second layer. These insights contribute to a deeper understanding of transformer behaviors in recursive tasks and offer potential avenues for improving AI alignment through mechanistic interpretability. Finally, we discuss the implications of our results for future work, including extensions to higher-dimensional recurrences and the exploration of polynomial sequences., Comment: 21 pages, 18 figures

In this paper, we consider teams of robots with heterogeneous skills (e.g., sensing and manipulation) tasked with collaborative missions described by Linear Temporal Logic (LTL) formulas. These LTL-encoded tasks require robots to apply their skills to specific regions and objects in a temporal and logical order. While existing temporal logic planning algorithms can synthesize correct-by-construction paths, they typically lack reactivity to unexpected failures of robot skills, which can compromise mission performance. This paper addresses this challenge by proposing a reactive LTL planning algorithm that adapts to unexpected failures during deployment. Specifically, the proposed algorithm reassigns sub-tasks to robots based on their functioning skills and locally revises team plans to accommodate these new assignments and ensure mission completion. The main novelty of the proposed algorithm is its ability to handle cases where mission completion becomes impossible due to limited functioning robots. Instead of reporting mission failure, the algorithm strategically prioritizes the most crucial sub-tasks and locally revises the team's plans, as per user-specified priorities, to minimize mission violations. We provide theoretical conditions under which the proposed framework computes the minimum violation task reassignments and team plans. We provide numerical and hardware experiments to demonstrate the efficiency of the proposed method.

This paper addresses the challenge of improving user experience on e-commerce platforms by enhancing product ranking relevant to users' search queries. Ambiguity and complexity of user queries often lead to a mismatch between the user's intent and retrieved product titles or documents. Recent approaches have proposed the use of Transformer-based models, which need millions of annotated query-title pairs during the pre-training stage, and this data often does not take user intent into account. To tackle this, we curate samples from existing datasets at eBay, manually annotated with buyer-centric relevance scores and centrality scores, which reflect how well the product title matches the users' intent. We introduce a User-intent Centrality Optimization (UCO) approach for existing models, which optimises for the user intent in semantic product search. To that end, we propose a dual-loss based optimisation to handle hard negatives, i.e., product titles that are semantically relevant but do not reflect the user's intent. Our contributions include curating challenging evaluation sets and implementing UCO, resulting in significant product ranking efficiency improvements observed for different evaluation metrics. Our work aims to ensure that the most buyer-centric titles for a query are ranked higher, thereby, enhancing the user experience on e-commerce platforms., Comment: EMNLP 2024: Industry track

The constant shifts in social and political contexts, driven by emerging social movements and political events, lead to new forms of hate content and previously unrecognized hate patterns that machine learning models may not have captured. Some recent literature proposes the data augmentation-based techniques to enrich existing hate datasets by incorporating samples that reveal new implicit hate patterns. This approach aims to improve the model's performance on out-of-domain implicit hate instances. It is observed, that further addition of more samples for augmentation results in the decrease of the performance of the model. In this work, we propose a Knowledge Transfer-driven Concept Refinement method that distills and refines the concepts related to implicit hate samples through novel prototype alignment and concept losses, alongside data augmentation based on concept activation vectors. Experiments with several publicly available datasets show that incorporating additional implicit samples reflecting new hate patterns through concept refinement enhances the model's performance, surpassing baseline results while maintaining cross-dataset generalization capabilities.\footnote{DISCLAIMER: This paper contains explicit statements that are potentially offensive.}, Comment: 11 pages, 4 figures, 2 algorithms, 5 tables

Visual servoing, the method of controlling robot motion through feedback from visual sensors, has seen significant advancements with the integration of optical flow-based methods. However, its application remains limited by inherent challenges, such as the necessity for a target image at test time, the requirement of substantial overlap between initial and target images, and the reliance on feedback from a single camera. This paper introduces Imagine2Servo, an innovative approach leveraging diffusion-based image editing techniques to enhance visual servoing algorithms by generating intermediate goal images. This methodology allows for the extension of visual servoing applications beyond traditional constraints, enabling tasks like long-range navigation and manipulation without predefined goal images. We propose a pipeline that synthesizes subgoal images grounded in the task at hand, facilitating servoing in scenarios with minimal initial and target image overlap and integrating multi-camera feedback for comprehensive task execution. Our contributions demonstrate a novel application of image generation to robotic control, significantly broadening the capabilities of visual servoing systems. Real-world experiments validate the effectiveness and versatility of the Imagine2Servo framework in accomplishing a variety of tasks, marking a notable advancement in the field of visual servoing., Comment: Published at 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Neural Radiance Fields (NeRFs) have shown significant promise in 3D scene reconstruction and novel view synthesis. In agricultural settings, NeRFs can serve as digital twins, providing critical information about fruit detection for yield estimation and other important metrics for farmers. However, traditional NeRFs are not robust to challenging lighting conditions, such as low-light, extreme bright light and varying lighting. To address these issues, this work leverages three different sensors: an RGB camera, an event camera and a thermal camera. Our RGB scene reconstruction shows an improvement in PSNR and SSIM by +2.06 dB and +8.3% respectively. Our cross-spectral scene reconstruction enhances downstream fruit detection by +43.0% in mAP50 and +61.1% increase in mAP50-95. The integration of additional sensors leads to a more robust and informative NeRF. We demonstrate that our multi-modal system yields high quality photo-realistic reconstructions under various tree canopy covers and at different times of the day. This work results in the development of a resilient NeRF, capable of performing well in visibly degraded scenarios, as well as a learnt cross-spectral representation, that is used for automated fruit detection., Comment: 7 pages, 5 figures

In this article, we propose a new method of generating single microwave photons in superconducting circuits. We theoretically show that pure single microwave photons can be generated on demand and tuned over a large frequency band by making use of Landau-Zener transitions under a rapid sweep of a control parameter. We devise a protocol that enables pulse-level control of the frequency of the emitted photon over two octaves, without requiring extensive calibration. Additionally, we make theoretical estimates of the generation efficiency, tunability, purity, and linewidth of the photons emitted using this method for both charge and flux qubit-based architectures. We also provide estimates of optimal device parameters for these architectures in order to realize the device., Comment: 7+7 pages, 12 figures, Comments welcome

This paper introduces an intelligent lecturing assistant (ILA) system that utilizes a knowledge graph to represent course content and optimal pedagogical strategies. The system is designed to support instructors in enhancing student learning through real-time analysis of voice, content, and teaching methods. As an initial investigation, we present a case study on lecture voice sentiment analysis, in which we developed a training set comprising over 3,000 one-minute lecture voice clips. Each clip was manually labeled as either engaging or non-engaging. Utilizing this dataset, we constructed and evaluated several classification models based on a variety of features extracted from the voice clips. The results demonstrate promising performance, achieving an F1-score of 90% for boring lectures on an independent set of over 800 test voice clips. This case study lays the groundwork for the development of a more sophisticated model that will integrate content analysis and pedagogical practices. Our ultimate goal is to aid instructors in teaching more engagingly and effectively by leveraging modern artificial intelligence techniques., Comment: Accepted in the 4th Workshop on Knowledge Graphs and Big Data @ IEEE Big Data Conference 2024

Complex dialog systems often use retrieved evidence to facilitate factual responses. Such RAG (Retrieval Augmented Generation) systems retrieve from massive heterogeneous data stores that are usually architected as multiple indexes or APIs instead of a single monolithic source. For a given query, relevant evidence needs to be retrieved from one or a small subset of possible retrieval sources. Complex queries can even require multi-step retrieval. For example, a conversational agent on a retail site answering customer questions about past orders will need to retrieve the appropriate customer order first and then the evidence relevant to the customer's question in the context of the ordered product. Most RAG Agents handle such Chain-of-Thought (CoT) tasks by interleaving reasoning and retrieval steps. However, each reasoning step directly adds to the latency of the system. For large models this latency cost is significant -- in the order of multiple seconds. Multi-agent systems may classify the query to a single Agent associated with a retrieval source, though this means that a (small) classification model dictates the performance of a large language model. In this work we present REAPER (REAsoning-based PlannER) - an LLM based planner to generate retrieval plans in conversational systems. We show significant gains in latency over Agent-based systems and are able to scale easily to new and unseen use cases as compared to classification-based planning. Though our method can be applied to any RAG system, we show our results in the context of a conversational shopping assistant.

Graph clustering is an important unsupervised learning technique for partitioning graphs with attributes and detecting communities. However, current methods struggle to accurately capture true community structures and intra-cluster relations, be computationally efficient, and identify smaller communities. We address these challenges by integrating coarsening and modularity maximization, effectively leveraging both adjacency and node features to enhance clustering accuracy. We propose a loss function incorporating log-determinant, smoothness, and modularity components using a block majorization-minimization technique, resulting in superior clustering outcomes. The method is theoretically consistent under the Degree-Corrected Stochastic Block Model (DC-SBM), ensuring asymptotic error-free performance and complete label recovery. Our provably convergent and time-efficient algorithm seamlessly integrates with graph neural networks (GNNs) and variational graph autoencoders (VGAEs) to learn enhanced node features and deliver exceptional clustering performance. Extensive experiments on benchmark datasets demonstrate its superiority over existing state-of-the-art methods for both attributed and non-attributed graphs., Comment: The first two authors contributed equally to this work