Search

Traditional evaluation of information retrieval (IR) systems relies on human-annotated relevance labels, which can be both biased and costly at scale. In this context, large language models (LLMs) offer an alternative by allowing us to directly prompt them to assign relevance labels for passages associated with each query. In this study, we explore alternative methods to directly prompt LLMs for assigned relevance labels, by exploring two hypotheses: Hypothesis 1 assumes that it is helpful to break down "relevance" into specific criteria - exactness, coverage, topicality, and contextual fit. We explore different approaches that prompt large language models (LLMs) to obtain criteria-level grades for all passages, and we consider various ways to aggregate criteria-level grades into a relevance label. Hypothesis 2 assumes that differences in linguistic style between queries and passages may negatively impact the automatic relevance label prediction. We explore whether improvements can be achieved by first synthesizing a summary of the passage in the linguistic style of a query, and then using this summary in place of the passage to assess its relevance. We include an empirical evaluation of our approaches based on data from the LLMJudge challenge run in Summer 2024, where our "Four Prompts" approach obtained the highest scores in Kendall's tau.

We give an overview of the science objectives and mission design of the Spectroscopic Time-Resolving Observatory for Broadband Energy X-rays (STROBE-X) observatory, which has been proposed as a NASA probe-class (~$1.5B) mission in response to the Astro2020 recommendation for an X-ray probe., Comment: 11 pages, 5 figures, accepted for publication in JATIS

Classical and wave properties of microlasers with the shape of a truncated pseudosphere are investigated through experiments and numerical simulations. These pseudosphere microlasers are surface-like organic microlasers with constant negative curvature, which were fabricated with high optical quality by direct laser writing. It is shown that they behave, in many ways, similar to two-dimensional flat disks, regardless of their different Gaussian curvature. Indeed, it is evidenced that, due to rotational symmetry, the pseudosphere is an integrable system with marginally stable dynamics., Comment: 16 pages, 16 figures

Understanding the interactions between nucleons in dense matter is one of the outstanding challenges of theoretical physics. Effective field theories have emerged as the dominant approach to address this problem at low energies, with many successful applications to the structure of nuclei and the properties of dense nucleonic matter. However, how far into the interior of neutron stars these interactions can describe dense matter is an open question. Here, we develop a framework that enables the inference of three-nucleon couplings in dense matter directly from astrophysical neutron-star observations. We apply this formalism to the LIGO/Virgo gravitational-wave event GW170817 and the X-ray measurements from NASA's Neutron-Star Interior Composition Explorer and establish direct constraints for the couplings that govern three-nucleon interactions in chiral effective field theory. Furthermore, we demonstrate how next-generation observations of a population of neutron-star mergers can offer stringent constraints on three-nucleon couplings, potentially at a level comparable to those from laboratory data. Our work directly connects the microscopic couplings in quantum field theories to macroscopic observations of neutron stars, providing a way to test the consistency between low-energy couplings inferred from terrestrial and astrophysical data., Comment: 22 pages and 4 figures

This paper presents an integration between DEMO (Design and Engineering Methodology for Organizations) and BPMN (Business Process Model and Notation). While BPMN is widely used for its intuitive, flow-based representation of business processes, it suffers from a lack of formal semantics, ambiguity, and limitations in modeling multi-party collaborations. In contrast, DEMO offers a theoretically robust, ontology-driven framework that focuses on abstracting the essential structure of business processes. A novel approach combining the rigor of DEMO's transaction patterns with the more practical, widely adopted BPMN framework is proposed and demonstrated. This integration allows for the benefits of DEMO's theoretical foundations to be utilized within BPMN diagrams, providing a more comprehensive and precise understanding of business processes. We argue that this combination enriches the modeling of business processes, providing a more coherent and reliable tool for both practitioners and researchers., Comment: 8 pages, 7 figures

Spatialized vibrotactile feedback systems deliver tactile information by placing multiple vibrotactile actuators on the body. As increasing numbers of actuators are required to adequately convey information in complicated applications, haptic designers find it difficult to create such systems due to limited scalability of existing toolkits. We propose VibraForge, an open-source vibrotactile toolkit that supports up to 128 vibrotactile actuators. Each actuator is encapsulated within a self-contained vibration unit and driven by its own microcontroller. By leveraging a chain-connection method, each unit receives independent vibration commands from a control unit, with fine-grained control over intensity and frequency. We also designed a GUI Editor to expedite the authoring of spatial vibrotactile patterns. Technical evaluations show that vibration units reliably reproduce audio waveforms with low-latency and high-bandwidth data communication. Case studies of phonemic tactile display, virtual reality fitness training, and drone teleoperation demonstrate the potential usage of VibraForge within different domains.

In the healthcare industry, researchers have been developing machine learning models to automate diagnosing patients with respiratory illnesses based on their breathing patterns. However, these models do not consider the demographic biases, particularly sex bias, that often occur when models are trained with a skewed patient dataset. Hence, it is essential in such an important industry to reduce this bias so that models can make fair diagnoses. In this work, we examine the bias in models used to detect breathing patterns of two major respiratory diseases, i.e., chronic obstructive pulmonary disease (COPD) and COVID-19. Using decision tree models trained with audio recordings of breathing patterns obtained from two open-source datasets consisting of 29 COPD and 680 COVID-19-positive patients, we analyze the effect of sex bias on the models. With a threshold optimizer and two constraints (demographic parity and equalized odds) to mitigate the bias, we witness 81.43% (demographic parity difference) and 71.81% (equalized odds difference) improvements. These findings are statistically significant., Comment: Accepted at 2024 IEEE-EMBS International Conference on Body Sensor Networks (IEEE BSN 2024)

Estimating the momentary level of participant's engagement is an important prerequisite for assistive systems that support human interactions. Previous work has addressed this task in within-domain evaluation scenarios, i.e. training and testing on the same dataset. This is in contrast to real-life scenarios where domain shifts between training and testing data frequently occur. With MultiMediate'24, we present the first challenge addressing multi-domain engagement estimation. As training data, we utilise the NOXI database of dyadic novice-expert interactions. In addition to within-domain test data, we add two new test domains. First, we introduce recordings following the NOXI protocol but covering languages that are not present in the NOXI training data. Second, we collected novel engagement annotations on the MPIIGroupInteraction dataset which consists of group discussions between three to four people. In this way, MultiMediate'24 evaluates the ability of approaches to generalise across factors such as language and cultural background, group size, task, and screen-mediated vs. face-to-face interaction. This paper describes the MultiMediate'24 challenge and presents baseline results. In addition, we discuss selected challenge solutions., Comment: arXiv admin note: text overlap with arXiv:2308.08256

Understanding human attention is crucial for vision science and AI. While many models exist for free-viewing, less is known about task-driven image exploration. To address this, we introduce CapMIT1003, a dataset with captions and click-contingent image explorations, to study human attention during the captioning task. We also present NevaClip, a zero-shot method for predicting visual scanpaths by combining CLIP models with NeVA algorithms. NevaClip generates fixations to align the representations of foveated visual stimuli and captions. The simulated scanpaths outperform existing human attention models in plausibility for captioning and free-viewing tasks. This research enhances the understanding of human attention and advances scanpath prediction models., Comment: arXiv admin note: substantial text overlap with arXiv:2305.12380

The remarkable ability of natural proteins to conduct electricity in the dry state over long distances remains largely inexplicable despite intensive research. In some cases, a (weakly) exponential length-attenuation, as in off-resonant tunneling transport, extends to thicknesses even beyond 10 nm. This report deals with such charge transport characteristics observed in self-assembled multilayers of the protein bacteriorhodopsin (bR). About 7.5 nm to 15.5 nm thick bR layers were prepared on conductive titanium nitride (TiN) substrates using aminohexylphosphonic acid and poly-diallyl-dimethylammonium electrostatic linkers. Using conical EGaIn top contacts, an intriguing, mono-exponential conductance attenuation as a function of the bR layer thickness with a small attenuation coefficient $\beta \approx 0.8 \space {\rm nm}^{-1}$ is measured at zero bias. Variable-temperature measurements using evaporated Ti/Au top contacts yield effective energy barriers of about 100 meV from fitting the data to tunneling, hopping, and carrier cascade transport models. The observed temperature-dependence is assigned to the protein-electrode interfaces. The transport length and temperature dependence of the current densities are consistent with tunneling through the protein-protein and protein-electrode interfaces, respectively. Importantly, our results call for new theoretical approaches to find the microscopic mechanism behind the remarkably efficient, long-range electron transport within bR.

We explore the hypothesis that poor compositional generalization in neural networks is caused by difficulties with learning effective routing. To solve this problem, we propose the concept of block-operations, which is based on splitting all activation tensors in the network into uniformly sized blocks and using an inductive bias to encourage modular routing and modification of these blocks. Based on this concept we introduce the Multiplexer, a new architectural component that enhances the Feed Forward Neural Network (FNN). We experimentally confirm that Multiplexers exhibit strong compositional generalization. On both a synthetic and a realistic task our model was able to learn the underlying process behind the task, whereas both FNNs and Transformers were only able to learn heuristic approximations. We propose as future work to use the principles of block-operations to improve other existing architectures.

Neural Networks are notoriously difficult to inspect. We introduce comgra, an open source python library for use with PyTorch. Comgra extracts data about the internal activations of a model and organizes it in a GUI (graphical user interface). It can show both summary statistics and individual data points, compare early and late stages of training, focus on individual samples of interest, and visualize the flow of the gradient through the network. This makes it possible to inspect the model's behavior from many different angles and save time by rapidly testing different hypotheses without having to rerun it. Comgra has applications for debugging, neural architecture design, and mechanistic interpretability. We publish our library through Python Package Index (PyPI) and provide code, documentation, and tutorials at https://github.com/FlorianDietz/comgra.

Urban parks provide significant health benefits by offering spaces and facilities for various recreational and leisure activities. However, the capacity of specific park spaces and elements to foster health remains underexamined. Traditional studies have focused on parks' size, greenery, and accessibility, often overlooking their ability to facilitate specific health-promoting activities. To address this gap, we propose a taxonomy consisting of six categories of health-promoting activities in parks: physical, mind-body, nature appreciation, environmental, social, and cultural. We estimate the capacity of parks in 35 global cities to promote health by establishing a lexicon linking park spaces and elements with specific health-promoting activities from our taxonomy. Using this lexicon, we collected data on elements and spaces in all parks in 35 cities from OpenStreetMap. Our analysis covers 23,477 parks with a total of 827,038 elements and spaces. By first comparing similarly sized parks across cities, we found that North American parks offer more spaces for physical activities, while European parks focus more on nature appreciation. Second, by scoring parks based on both elements and spaces, we investigated the variability in their health-promoting potential. We found the most uniform provision across parks for physical activities and the highest disparities regarding social activities. Additionally, parks offering a variety of activities are usually located in city centers, while offerings diminish in parks towards the suburbs. Lastly, we identified significant inequalities in park standards across cities, regardless of their continental location: Tokyo and Paris offer the most uniform park standards, while Copenhagen and Rio de Janeiro exhibit the most pronounced disparities. Our study provides insights for making urban parks more equitable, engaging, and health-promoting., Comment: 29 pages main paper, 10 pages appendix

The $^{246}$Cm(n,$\gamma$) and $^{248}$Cm(n,$\gamma$) cross-sections have been measured at the Experimental Area 2 (EAR2) of the n_TOF facility at CERN with three C$_6$D$_6$ detectors. This measurement is part of a collective effort to improve the capture cross-section data for Minor Actinides (MAs), which are required to estimate the production and transmutation rates of these isotopes in light water reactors and innovative reactor systems. In particular, the neutron capture in $^{246}$Cm and $^{248}$Cm open the path for the formation of other Cm isotopes and heavier elements such as Bk and Cf and the knowledge of (n,$\gamma$) cross-sections of these Cm isotopes plays an important role in the transport, transmutation and storage of the spent nuclear fuel. The reactions $^{246}$Cm(n,$\gamma$) and $^{248}$Cm(n,$\gamma$) have been the two first capture measurements analyzed at n_TOF EAR2. Until this experiment and two recent measurements performed at J-PARC, there was only one set of data of the capture cross-sections of $^{246}$Cm and $^{248}$Cm, that was obtained in 1969 in an underground nuclear explosion experiment. In the measurement at n_TOF a total of 13 resonances of $^{246}$Cm between 4 and 400 eV and 5 of $^{248}$Cm between 7 and 100 eV have been identified and fitted. The radiative kernels obtained for $^{246}$Cm are compatible with JENDL-5, but some of them are not with JENDL-4, which has been adopted by JEFF-3.3 and ENDF/B-VIII.0. The radiative kernels obtained for the first three $^{248}$Cm resonances are compatible with JENDL-5, however, the other two are not compatible with any other evaluation and are 20% and 60% larger than JENDL-5.

Mixed-type time series (MTTS) is a bimodal data type that is common in many domains, such as healthcare, finance, environmental monitoring, and social media. It consists of regularly sampled continuous time series and irregularly sampled categorical event sequences. The integration of both modalities through multimodal fusion is a promising approach for processing MTTS. However, the question of how to effectively fuse both modalities remains open. In this paper, we present a comprehensive evaluation of several deep multimodal fusion approaches for MTTS forecasting. Our comparison includes three fusion types (early, intermediate, and late) and five fusion methods (concatenation, weighted mean, weighted mean with correlation, gating, and feature sharing). We evaluate these fusion approaches on three distinct datasets, one of which was generated using a novel framework. This framework allows for the control of key data properties, such as the strength and direction of intermodal interactions, modality imbalance, and the degree of randomness in each modality, providing a more controlled environment for testing fusion approaches. Our findings show that the performance of different fusion approaches can be substantially influenced by the direction and strength of intermodal interactions. The study reveals that early and intermediate fusion approaches excel at capturing fine-grained and coarse-grained cross-modal features, respectively. These findings underscore the crucial role of intermodal interactions in determining the most effective fusion strategy for MTTS forecasting.

Silicon carbide (SiC) is a semiconductor used in quantum information processing, microelectromechanical systems, photonics, power electronics, and harsh environment sensors. However, its high temperature stability, high breakdown voltage, wide bandgap, and high mechanical strength are accompanied by a chemical inertness which makes complex micromachining difficult. Photoelectrochemical etching is a simple, rapid means of wet processing SiC, including the use of dopant selective etch stops that take advantage of mature SiC homoepitaxy. However, dopant selective photoelectrochemical etching typically relies on highly doped material, which poses challenges for device applications such as quantum defects and photonics that benefit from low doping to produce robust emitter properties and high optical transparency. In this work, we develop a new, selective photoelectrochemical etching process that relies not on high doping but on the electrical depletion of a fabricated diode structure, allowing the selective etching of an n-doped substrate wafer versus an undoped epitaxial ($N_a=1(10)^{14}cm^{-3}$) device layer. We characterize the photo-response and photoelectrochemical etching behavior of the diode under bias and use those insights to suspend large ($>100\mu m^2$) undoped membranes of SiC. We further characterize the compatibility of membranes with quantum emitters, performing comparative spin spectroscopy between undoped and highly doped membrane structures, finding the use of undoped material improves ensemble spin lifetime by $>3x$. This work enables the fabrication of high-purity suspended thin films suitable for scalable photonics, mechanics, and quantum technologies in SiC., Comment: 12 pages, 4 figures

This resource paper addresses the challenge of evaluating Information Retrieval (IR) systems in the era of autoregressive Large Language Models (LLMs). Traditional methods relying on passage-level judgments are no longer effective due to the diversity of responses generated by LLM-based systems. We provide a workbench to explore several alternative evaluation approaches to judge the relevance of a system's response that incorporate LLMs: 1. Asking an LLM whether the response is relevant; 2. Asking the LLM which set of nuggets (i.e., relevant key facts) is covered in the response; 3. Asking the LLM to answer a set of exam questions with the response. This workbench aims to facilitate the development of new, reusable test collections. Researchers can manually refine sets of nuggets and exam questions, observing their impact on system evaluation and leaderboard rankings. Resource available at https://github.com/TREMA-UNH/autograding-workbench, Comment: 10 pages. To appear in the Resource & Reproducibility Track of SIGIR 2024

Much of early literacy education happens at home with caretakers reading books to young children. Prior research demonstrates how having dialogue with children during co-reading can develop critical reading readiness skills, but most adult readers are unsure if and how to lead effective conversations. We present ContextQ, a tablet-based reading application to unobtrusively present auto-generated dialogic questions to caretakers to support this dialogic reading practice. An ablation study demonstrates how our method of encoding educator expertise into the question generation pipeline can produce high-quality output; and through a user study with 12 parent-child dyads (child age: 4-6), we demonstrate that this system can serve as a guide for parents in leading contextually meaningful dialogue, leading to significantly more conversational turns from both the parent and the child and deeper conversations with connections to the child's everyday life., Comment: ACM Interaction Design and Children (IDC) 2024

We study fluctuation properties in the energy spectra of finite-size honeycomb lattices, graphene billiards, subject to the Haldane-model onsite potential and next-nearest neighbor interaction at critical points, referred to as Haldane graphene billiards in the following. The billiards had the shapes of a rectangular billiard with integrable dynamics, one with chaotic dynamics, and one whose shape has, in addition, threefold rotational symmetry. It had been shown that the spectral properties of the graphene billiards coincide with those of the nonrelativistic quantum billiard with the corresponding shape, both at the band edges and in the region of low energy excitations around the Dirac points at zero energy. There, the dispersion relation is linear and, accordingly, the spectrum is described by the same relativistic Dirac equation for massless half-spin particles as relativistic neutrino billiards, whose spectral properties agree with those of nonrelativistic quantum billiards with violated time-reversal invariance. Deviations from the expected behavior are attributed to differing boundary conditions and backscattering at the boundary, which leads to a mixing of valley states corresponding to the two Dirac points, that are mapped into each other through time reversal. We employ a Haldane model to introduce a gap at one of the two Dirac points so that backscattering is suppressed in the energy region of the gap and demonstrate that there the correlations in the spectra comply with those of the neutrino billiard of the corresponding shape.

We present numerical results for the Rosenzweig Porter model for all symmetry classes of the Dyson threefold way. We analyzed the fluctuation properties in the eigenvalue spectra, and compared them with existing and new analytical results. Based on these results we propose characteristics of the spectral properties as measures to explore the transition from Poisson to Wigner Dyson WD statistics. Furthermore, we performed thorough studies of the properties of the eigenvectors in terms of the fractal dimensions, the Kullback Leibler KL divergences and the fidelity susceptibility. The ergodic and Anderson transitions take place at the same parameter values and a finite size scaling analysis of the KL divergences at the transitions yields the same critical exponents for all three WD classes, thus indicating superuniversality of these transitions.

This paper extends the Finite Elements with Switch Detection and Jumps (FESD-J) [1] method to problems of rigid body dynamics involving patch contacts. The FESD-J method is a high accuracy discretization scheme suitable for use in direct optimal control of nonsmooth mechanical systems. It detects dynamic switches exactly in time and, thereby, maintains the integration order of the underlying Runge- Kutta (RK) method. This is in contrast to commonly used time-stepping methods which only achieve first-order accuracy. Considering rigid bodies with possible patch contacts results in nondifferentiable signed distance functions (SDF), which introduces additional nonsmoothness into the dynamical system. In this work, we utilize so-called equivalent contact points (ECP), which parameterize force and impulse distributions on contact patches by evaluation at single points. We embed a nondifferentiable SDF into a complementarity Lagrangian system (CLS) and show that the determined ECP are well-defined. We then extend the FESD-J discretization to the considered CLS such that its integration accuracy is maintained. The functionality of the method is illustrated for both a simulation and an optimal control example., Comment: Shortened version submitted to 2024 Conference on Decision and Control (CDC)

In the field of affective computing, where research continually advances at a rapid pace, the demand for user-friendly tools has become increasingly apparent. In this paper, we present the AffectToolbox, a novel software system that aims to support researchers in developing affect-sensitive studies and prototypes. The proposed system addresses the challenges posed by existing frameworks, which often require profound programming knowledge and cater primarily to power-users or skilled developers. Aiming to facilitate ease of use, the AffectToolbox requires no programming knowledge and offers its functionality to reliably analyze the affective state of users through an accessible graphical user interface. The architecture encompasses a variety of models for emotion recognition on multiple affective channels and modalities, as well as an elaborate fusion system to merge multi-modal assessments into a unified result. The entire system is open-sourced and will be publicly available to ensure easy integration into more complex applications through a well-structured, Python-based code base - therefore marking a substantial contribution toward advancing affective computing research and fostering a more collaborative and inclusive environment within this interdisciplinary field.

In the landscape of technological innovation, autonomous racing is a dynamic and challenging domain that not only pushes the limits of technology, but also plays a crucial role in advancing and fostering a greater acceptance of autonomous systems. This paper thoroughly explores challenges and advances in autonomous racing vehicle design and performance, focusing on Roborace and the Indy Autonomous Challenge (IAC). This review provides a detailed analysis of sensor setups, architectural nuances, and test metrics on these cutting-edge platforms. In Roborace, the evolution from Devbot 1.0 to Robocar and Devbot 2.0 is detailed, revealing insights into sensor configurations and performance outcomes. The examination extends to the IAC, which is dedicated to high-speed self-driving vehicles, emphasizing developmental trajectories and sensor adaptations. By reviewing these platforms, the analysis provides valuable insight into autonomous driving racing, contributing to a broader understanding of sensor architectures and the challenges faced. This review supports future advances in full-scale autonomous racing technology.

Current IR evaluation is based on relevance judgments, created either manually or automatically, with decisions outsourced to Large Language Models (LLMs). We offer an alternative paradigm, that never relies on relevance judgments in any form. Instead, a text is defined as relevant if it contains information that enables the answering of key questions. We use this idea to design the EXAM Answerability Metric to evaluate information retrieval/generation systems for their ability to provide topically relevant information. We envision the role of a human judge to edit and define an exam question bank that will test for the presence of relevant information in text. We support this step by generating an initial set of exam questions. In the next phase, an LLM-based question answering system will automatically grade system responses by tracking which exam questions are answerable with which system responses. We propose two evaluation measures, the recall-oriented EXAM Cover metric, and the precision-oriented EXAM Qrels metric, the latter which can be implemented with trec_eval. This paradigm not only allows for the expansion of the exam question set post-hoc but also facilitates the ongoing evaluation of future information systems, whether they focus on retrieval, generation, or both., Comment: 10 pages

We report on high-precision measurements that were performed with superconducting waveguide networks with the geometry of a tetrahedral and a honeycomb graph. They consist of junctions of valency three that connect straight rectangular waveguides of incommensurable lengths. The experiments were performed in the frequency range of a single transversal mode, where the associated Helmholtz equation is effectively one dimensional and waveguide networks may serve as models of quantum graphs with the joints and waveguides corresponding to the vertices and bonds. The tetrahedral network comprises T junctions, while the honeycomb network exclusively consists of Y junctions, that join waveguides with relative angles 90 degree and 120 degree, respectively. We demonstrate that the vertex scattering matrix, which describes the propagation of the modes through the junctions strongly depends on frequency and is non-symmetric at a T junction and thus differs from that of a quantum graph with Neumann boundary conditions at the vertices. On the contrary, at a Y junction, similarity can be achieved in a certain frequeny range. We investigate the spectral properties of closed waveguide networks and fluctuation properties of the scattering matrix of open ones and find good agreement with random matrix theory predictions for the honeycomb waveguide graph.

Time series forecasting is a challenging task due to the existence of complex and dynamic temporal dependencies. This can lead to incorrect predictions by even the best forecasting models. Using more training data is one way to improve the accuracy, but this source is often limited. In contrast, we are building on successful denoising approaches for image generation by advocating for an end-to-end forecasting and denoising paradigm. We propose an end-to-end forecast-blur-denoise forecasting framework by encouraging a division of labors between the forecasting and the denoising models. The initial forecasting model is directed to focus on accurately predicting the coarse-grained behavior, while the denoiser model focuses on capturing the fine-grained behavior that is locally blurred by integrating a Gaussian Process model. All three parts are interacting for the best end-to-end performance. Our extensive experiments demonstrate that our proposed approach is able to improve the forecasting accuracy of several state-of-the-art forecasting models as well as several other denoising approaches., Comment: 10 pages