Search

In this paper we propose the Hatching-Box, a novel imaging and analysis system to automatically monitor and quantify the developmental behavior of Drosophila in standard rearing vials and during regular rearing routines, rendering explicit experiments obsolete. This is achieved by combining custom tailored imaging hardware with dedicated detection and tracking algorithms, enabling the quantification of larvae, filled/empty pupae and flies over multiple days. Given the affordable and reproducible design of the Hatching-Box in combination with our generic client/server-based software, the system can easily be scaled to monitor an arbitrary amount of rearing vials simultaneously. We evaluated our system on a curated image dataset comprising nearly 470,000 annotated objects and performed several studies on real world experiments. We successfully reproduced results from well-established circadian experiments by comparing the eclosion periods of wild type flies to the clock mutants $\textit{per}^{short}$, $\textit{per}^{long}$ and $\textit{per}^0$ without involvement of any manual labor. Furthermore we show, that the Hatching-Box is able to extract additional information about group behavior as well as to reconstruct the whole life-cycle of the individual specimens. These results not only demonstrate the applicability of our system for long-term experiments but also indicate its benefits for automated monitoring in the general cultivation process., Comment: 17 pages, 6 figures

Advancements in tracking algorithms have empowered nascent applications across various domains, from steering autonomous vehicles to guiding robots to enhancing augmented reality experiences for users. However, these algorithms are application-specific and do not work across applications with different types of motion; even a tracking algorithm designed for a given application does not work in scenarios deviating from highly standard conditions. For example, a tracking algorithm designed for robot navigation inside a building will not work for tracking the same robot in an outdoor environment. To demonstrate this problem, we evaluate the performance of the state-of-the-art tracking methods across various applications and scenarios. To inform our analysis, we first categorize algorithmic, environmental, and locomotion-related challenges faced by tracking algorithms. We quantitatively evaluate the performance using multiple tracking algorithms and representative datasets for a wide range of Internet of Things (IoT) and Extended Reality (XR) applications, including autonomous vehicles, drones, and humans. Our analysis shows that no tracking algorithm works across different applications and scenarios within applications. Ultimately, using the insights generated from our analysis, we discuss multiple approaches to improving the tracking performance using input data characterization, leveraging intermediate information, and output evaluation.

Autonomous robots, autonomous vehicles, and humans wearing mixed-reality headsets require accurate and reliable tracking services for safety-critical applications in dynamically changing real-world environments. However, the existing tracking approaches, such as Simultaneous Localization and Mapping (SLAM), do not adapt well to environmental changes and boundary conditions despite extensive manual tuning. On the other hand, while deep learning-based approaches can better adapt to environmental changes, they typically demand substantial data for training and often lack flexibility in adapting to new domains. To solve this problem, we propose leveraging the neurosymbolic program synthesis approach to construct adaptable SLAM pipelines that integrate the domain knowledge from traditional SLAM approaches while leveraging data to learn complex relationships. While the approach can synthesize end-to-end SLAM pipelines, we focus on synthesizing the feature extraction module. We first devise a domain-specific language (DSL) that can encapsulate domain knowledge on the important attributes for feature extraction and the real-world performance of various feature extractors. Our neurosymbolic architecture then undertakes adaptive feature extraction, optimizing parameters via learning while employing symbolic reasoning to select the most suitable feature extractor. Our evaluations demonstrate that our approach, neurosymbolic Feature EXtraction (nFEX), yields higher-quality features. It also reduces the pose error observed for the state-of-the-art baseline feature extractors ORB and SIFT by up to 90% and up to 66%, respectively, thereby enhancing the system's efficiency and adaptability to novel environments., Comment: 8 pages, 6 figures, and 5 tables. Published at the 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Corresponding author: Yasra Chandio (ychandio@umass.edu)

Contrastive learning (CL) has emerged as a potent tool for building meaningful latent representations of galaxy properties across a broad spectrum of wavelengths, ranging from optical and infrared to radio frequencies. These representations facilitate a variety of downstream tasks, including galaxy classification, similarity searches, and parameter estimation, which is why they are often referred to as foundation models. In this study, we employ CL on the latest extended DR from CALIFA survey, which encompasses 895 galaxies with enhanced spatial resolution. We demonstrate that CL can be applied to IFU surveys, even with small training sets, to meaningful embedding where galaxies are well-separated based on their physical properties. We discover that the strongest correlations in the embedding space are observed with the EW of Ha morphology, stellar metallicity, age, stellar surface mass density, the [NII]/Ha ratio, and stellar mass, in descending order of correlation strength. Additionally, we illustrate the feasibility of unsupervised separation of galaxy populations along the SFMS, successfully identifying the BC and the RS in a two-cluster scenario, and the GV population in a three-cluster scenario. Our findings indicate that galaxy luminosity profiles have minimal impact on the construction of the embedding space, suggesting that morphology and spectral features play a more significant role in distinguishing between galaxy populations. Moreover, we explore the use of CL for detecting variations in galaxy population distributions across different environments, including voids, clusters, filaments and walls. Nonetheless, we acknowledge the limitations of the CL and our specific training set in detecting subtle differences in galaxy properties, such as the presence of an AGN or other minor scale variations that exceed the scope of primary parameters like stellar mass or morphology., Comment: 21 pages, 12 figures

Grape cluster architecture and compactness are complex traits influencing disease susceptibility, fruit quality, and yield. Evaluation methods for these traits include visual scoring, manual methodologies, and computer vision, with the latter being the most scalable approach. Most of the existing computer vision approaches for processing cluster images often rely on conventional segmentation or machine learning with extensive training and limited generalization. The Segment Anything Model (SAM), a novel foundation model trained on a massive image dataset, enables automated object segmentation without additional training. This study demonstrates out-of-the-box SAM's high accuracy in identifying individual berries in 2D cluster images. Using this model, we managed to segment approximately 3,500 cluster images, generating over 150,000 berry masks, each linked with spatial coordinates within their clusters. The correlation between human-identified berries and SAM predictions was very strong (Pearson r2=0.96). Although the visible berry count in images typically underestimates the actual cluster berry count due to visibility issues, we demonstrated that this discrepancy could be adjusted using a linear regression model (adjusted R2=0.87). We emphasized the critical importance of the angle at which the cluster is imaged, noting its substantial effect on berry counts and architecture. We proposed different approaches in which berry location information facilitated the calculation of complex features related to cluster architecture and compactness. Finally, we discussed SAM's potential integration into currently available pipelines for image generation and processing in vineyard conditions.

Extreme Emission Line Galaxies (EELGs) stand as remarkable objects due to their extremely metal poor environment and intense star formation. Considered as local analogues of high-redshift galaxies in the peak of their star-forming activity, they offer insights into conditions prevalent during the early Universe. Assessment of their stellar and gas properties is, therefore, of critical importance, which requires the assembly of a considerable sample, comprehending a broad redshift range. The Javalambre-Physics of the Accelerating Universe Astrophysical Survey (JPAS) plays a significant role in assembling such a sample, encompassing approximately 8000 deg2 and employing 54 narrow-band optical filters. The present work describes the development and subsequent application of the tools that will be employed in the forthcoming JPAS spectrophotometric data, allowing for the massive and automated characterization of EELGs that are expected to be identified. This fully automated pipeline (requiring only the object coordinates from users) constructs Spectral Energy Distributions (SEDs) by retrieving virtually all the available multi-wavelength photometric data archives, employs SED fitting tools and identifies optical emission lines. It was applied to the sample of extreme line emitters identified in the miniJPAS Survey, and its derived physical properties such as stellar mass and age, coupled with fundamental relations, mirror results obtained through spectral modeling of SDSS spectra. Thorough testing using galaxies with documented photometric measurements across different wavelengths confirmed the pipeline's accuracy, demonstrating its capability for automated analysis of sources with varying characteristics, spanning brightness, morphology, and redshifts. The modular nature of this pipeline facilitates any addition from the user., Comment: 15 pages, 9 Figures, accepted for publication in MNRAS

While IoT sensors in physical spaces have provided utility and comfort in our lives, their instrumentation in private and personal spaces has led to growing concerns regarding privacy. The existing notion behind IoT privacy is that the sensors whose data can easily be understood and interpreted by humans (such as cameras) are more privacy-invasive than sensors that are not human-understandable, such as RF (radio-frequency) sensors. However, given recent advancements in machine learning, we can not only make sensitive inferences on RF data but also translate between modalities. Thus, the existing notions of privacy for IoT sensors need to be revisited. In this paper, our goal is to understand what factors affect the privacy notions of a non-expert user (someone who is not well-versed in privacy concepts). To this regard, we conduct an online study of 162 participants from the USA to find out what factors affect the privacy perception of a user regarding an RF-based device or a sensor. Our findings show that a user's perception of privacy not only depends upon the data collected by the sensor but also on the inferences that can be made on that data, familiarity with the device and its form factor as well as the control a user has over the device design and its data policies. When the data collected by the sensor is not human-interpretable, it is the inferences that can be made on the data and not the data itself that users care about when making informed decisions regarding device privacy.

We propose a conjecture extending the classical construction of elliptic units to complex cubic number fields $K$. The conjecture concerns special values of the elliptic gamma function, a holomorphic function of three complex variables arising in mathematical physics whose transformation properties under $\mathrm{SL}_3(\mathbf{Z})$ were studied by Felder and Varchenko in the early 2000s. Using this function we construct complex numbers that we conjecture to be units in narrow ray class fields of $K$. We also propose a reciprocity law for the action of the Galois group on these units in the style of Shimura. To support our conjecture we offer numerical evidence and also prove a new type of Kronecker limit formula relating the logarithm of the modulus of these complex numbers to the derivatives at $s = 0$ of partial zeta functions of $K$. Our constructions unveil the role played by the elliptic gamma function in Hilbert's twelfth problem for complex cubic fields., Comment: 44 pages, 1 figure

Symmetry principles have proven important in physics, deep learning and geometry, allowing for the reduction of complicated systems to simpler, more comprehensible models that preserve the system's features of interest. Biological systems often show a high level of complexity and consist of a high number of interacting parts. Using symmetry fibrations, the relevant symmetries for biological 'message-passing' networks, we reduced the gene regulatory networks of E. coli and B. subtilis bacteria in a way that preserves information flow and highlights the computational capabilities of the network. Nodes that share isomorphic input trees are grouped into equivalence classes called fibers, whereby genes that receive signals with the same 'history' belong to one fiber and synchronize. We further reduce the networks to its computational core by removing 'dangling ends' via k-core decomposition. The computational core of the network consists of a few strongly connected components in which signals can cycle while signals are transmitted between these 'information vortices' in a linear feed-forward manner. These components are in charge of decision making in the bacterial cell by employing a series of genetic toggle-switch circuits that store memory, and oscillator circuits. These circuits act as the central computation machine of the network, whose output signals then spread to the rest of the network.

The reconfigurable intelligent surface (RIS) technology allows one to engineer spatial diversity in complex cellular networks. This paper provides a framework for the system-level performance assessment of RIS-assisted networks and in particular downlink coverage probability and ergodic rate. To account for the inherent randomness in the spatial deployments of base stations (BSs) and RISs, we model the placements of the RISs as point processes (PPs) conditioned on the associated BSs, which are modeled by a Poisson point process (PPP). These RIS PPs can be adapted based on the deployment strategy. We focus on modeling the RISs as a Mat\'ern cluster process (MCP), where each RIS cluster is a finite PPP with support a disc centered on the association BS. We assume that the system uses the orthogonal frequency division multiplexing (OFDM) technique to exploit the multipath diversity provided by RISs. The coverage probability and the ergodic rate can be evaluated when RISs operate as batched powerless beamformers. The resulting analytical expressions provide a general methodology to evaluate the impact of key RIS-related parameters, such as the batch size and the density of RISs, on system-level performance. To demonstrate the framework's broad applicability, we also analyze a RIS placement variant where RISs are deployed around coverage holes. Numerical evaluations of the analytical expressions and Monte-Carlo simulations jointly validate the proposed analytical approach and provide valuable insights into the design of future RIS-assisted cellular networks.

First we give conditions on a Banach lattice $E$ so that an operator $T$ from $E$ to any Banach space is disjoint $p$-convergent if and only if $T$ is almost Dunford-Pettis. Then we study when adjoints of positive operators between Banach lattices are disjoint $p$-convergent. For instance, we prove that the following conditions are equivalent for all Banach lattices $E$ and $F$: (i) A positive operator $T \colon E \to F$ is almost weak $p$-convergent if and only if $T^*$ is disjoint $p$-convergent; (ii) $E^*$ has order continuous norm or $F^*$ has the positive Schur property of order $p$. Very recent results are improved, examples are given and applications of the main results are provided., Comment: 15 pages

Camera-based autonomous systems that emulate human perception are increasingly being integrated into safety-critical platforms. Consequently, an established body of literature has emerged that explores adversarial attacks targeting the underlying machine learning models. Adapting adversarial attacks to the physical world is desirable for the attacker, as this removes the need to compromise digital systems. However, the real world poses challenges related to the "survivability" of adversarial manipulations given environmental noise in perception pipelines and the dynamicity of autonomous systems. In this paper, we take a sensor-first approach. We present EvilEye, a man-in-the-middle perception attack that leverages transparent displays to generate dynamic physical adversarial examples. EvilEye exploits the camera's optics to induce misclassifications under a variety of illumination conditions. To generate dynamic perturbations, we formalize the projection of a digital attack into the physical domain by modeling the transformation function of the captured image through the optical pipeline. Our extensive experiments show that EvilEye's generated adversarial perturbations are much more robust across varying environmental light conditions relative to existing physical perturbation frameworks, achieving a high attack success rate (ASR) while bypassing state-of-the-art physical adversarial detection frameworks. We demonstrate that the dynamic nature of EvilEye enables attackers to adapt adversarial examples across a variety of objects with a significantly higher ASR compared to state-of-the-art physical world attack frameworks. Finally, we discuss mitigation strategies against the EvilEye attack., Comment: 15 pages, 11 figures

Existing approaches for autonomous control of pan-tilt-zoom (PTZ) cameras use multiple stages where object detection and localization are performed separately from the control of the PTZ mechanisms. These approaches require manual labels and suffer from performance bottlenecks due to error propagation across the multi-stage flow of information. The large size of object detection neural networks also makes prior solutions infeasible for real-time deployment in resource-constrained devices. We present an end-to-end deep reinforcement learning (RL) solution called Eagle to train a neural network policy that directly takes images as input to control the PTZ camera. Training reinforcement learning is cumbersome in the real world due to labeling effort, runtime environment stochasticity, and fragile experimental setups. We introduce a photo-realistic simulation framework for training and evaluation of PTZ camera control policies. Eagle achieves superior camera control performance by maintaining the object of interest close to the center of captured images at high resolution and has up to 17% more tracking duration than the state-of-the-art. Eagle policies are lightweight (90x fewer parameters than Yolo5s) and can run on embedded camera platforms such as Raspberry PI (33 FPS) and Jetson Nano (38 FPS), facilitating real-time PTZ tracking for resource-constrained environments. With domain randomization, Eagle policies trained in our simulator can be transferred directly to real-world scenarios., Comment: 20 pages, IoTDI

This paper is part of large effort within the J-PAS collaboration that aims to classify point-like sources in miniJPAS, which were observed in 60 optical bands over $\sim$ 1 deg$^2$ in the AEGIS field. We developed two algorithms based on artificial neural networks (ANN) to classify objects into four categories: stars, galaxies, quasars at low redshift ($z < 2.1)$, and quasars at high redshift ($z \geq 2.1$). As inputs, we used miniJPAS fluxes for one of the classifiers (ANN$_1$) and colours for the other (ANN$_2$). The ANNs were trained and tested using mock data in the first place. We studied the effect of augmenting the training set by creating hybrid objects, which combines fluxes from stars, galaxies, and quasars. Nevertheless, the augmentation processing did not improve the score of the ANN. We also evaluated the performance of the classifiers in a small subset of the SDSS DR12Q superset observed by miniJPAS. In the mock test set, the f1-score for quasars at high redshift with the ANN$_1$ (ANN$_2$) are $0.99$ ($0.99$), $0.93$ ($0.92$), and $0.63$ ($0.57$) for $17 < r \leq 20$, $20 < r \leq 22.5$, and $22.5 < r \leq 23.6$, respectively, where $r$ is the J-PAS rSDSS band. In the case of low-redshift quasars, galaxies, and stars, we reached $0.97$ ($0.97$), $0.82$ ($0.79$), and $0.61$ ($0.58$); $0.94$ ($0.94$), $0.90$ ($0.89$), and $0.81$ ($0.80$); and $1.0$ ($1.0$), $0.96$ ($0.94$), and $0.70$ ($0.52$) in the same r bins. In the SDSS DR12Q superset miniJPAS sample, the weighted f1-score reaches 0.87 (0.88) for objects that are mostly within $20 < r \leq 22.5$. Finally, we estimate the number of point-like sources that are quasars, galaxies, and stars in miniJPAS.

Model-based methods are widely used for reconstruction in compressed sensing (CS) magnetic resonance imaging (MRI), using regularizers to describe the images of interest. The reconstruction process is equivalent to solving a composite optimization problem. Accelerated proximal methods (APMs) are very popular approaches for such problems. This paper proposes a complex quasi-Newton proximal method (CQNPM) for the wavelet and total variation based CS MRI reconstruction. Compared with APMs, CQNPM requires fewer iterations to converge but needs to compute a more challenging proximal mapping called weighted proximal mapping (WPM). To make CQNPM more practical, we propose efficient methods to solve the related WPM. Numerical experiments on reconstructing non-Cartesian MRI data demonstrate the effectiveness and efficiency of CQNPM., Comment: 26 pages, 26 figures

Many authors have constructed different, but related, linear group cocycles that are usually referred to as ``Eisenstein cocycles.'' The main goal of this work is to describe a topological construction that is a common source for all these cocycles. One interesting feature of this construction is that, starting from a purely topological class, it leads to the algebraic world of meromorphic forms on hyperplane complements in $n$-fold products of either the (complex) additive group, the multiplicative group or a (family of) elliptic curve(s). This yields the construction of three types of ``Sczech cocycles.'', Comment: 141 pages, 3 figures, the text is in French. Submitted as a CRM Monograph

We consider arbitrary polarized variations of Hodge structure of weight two and $h^{2,0}=1$ over a non--singular complex algebraic curve $S$ and analyze the boundary behaviour of the associated Kudla--Millson theta series using Schmid's theorems on degenerations of Hodge structure. This allows us to prove that this theta series is always integrable over $S$ and to describe explicitly the non-holomorphic part of the Kudla--Millson generating series in terms of the mixed Hodge structures at infinity.

Although organizations are continuously making concerted efforts to harden their systems against network attacks by air-gapping critical systems, attackers continuously adapt and uncover covert channels to exfiltrate data from air-gapped systems. For instance, attackers have demonstrated the feasibility of exfiltrating data from a computer sitting in a Faraday cage by exfiltrating data using magnetic fields. Although a large body of work has recently emerged highlighting various physical covert channels, these attacks have mostly targeted open-loop cyber-physical systems where the covert channels exist on physical channels that are not being monitored by the victim. Network architectures such as fog computing push sensitive data to cyber-physical edge devices--whose physical side channels are typically monitored via state estimation. In this paper, we formalize covert data exfiltration that uses existing cyber-physical models and infrastructure of individual devices to exfiltrate data in a stealthy manner, i.e., we propose a method to circumvent cyber-physical state estimation intrusion detection techniques while exfiltrating sensitive data from the network. We propose a generalized model for encoding and decoding sensitive data within cyber-physical control loops. We evaluate our approach on a distributed IoT network that includes computation nodes residing on physical drones as well as on an industrial control system for the control of a robotic arm. Unlike prior works, we formalize the constraints of covert cyber-physical channel exfiltration in the presence of a defender performing state estimation.

We prove several results establishing conditions on the Banach lattices E_1,..., E_m and F so that the Aron-Berner extensions of (separately) almost Dunford-Pettis m-linear operators from E_1 x ... x E_m to F are (separately) almost Dunford-Pettis. Illustrative examples are provided., Comment: 17 pages

Technological advances in holography, robotics, and 3D printing are starting to realize the vision of a holodeck. These immersive 3D displays must address user safety from the start to be viable. A holodeck's safety challenges are novel because its applications will involve explicit physical interactions between humans and synthesized 3D objects and experiences in real-time. This pioneering paper first proposes research directions for modeling safety in future holodeck applications from traditional physical human-robot interaction modeling. Subsequently, we propose a test-bed to enable safety validation of physical human-robot interaction based on existing augmented reality and virtual simulation technology., Comment: Appeared in the CHI Conference on Human Factors in Computing Systems (CHI'22), April 29-May 5, 2022, New Orleans, LA, USA. ACM, New York, NY, 3 pages

First we characterize the Banach lattices E whose biduals have the positive Schur property by means of second adjoints of operators on E being almost Dunford-Pettis. Next we extend some known results concerning conditions on the Banach lattices E and F under which the adjoint T* and the second adjoint T** of any positive almost Dunford-Pettis operator T from E to F are almost Dunford-Pettis. Finally, we prove when T* and T** are almost Dunford-Pettis for any (non necessarily almost Dunford-Pettis) T that is either bounded, regular, order bounded or weakly compact., Comment: 13 pages

Background: Diabetes is associated with obesity, poor glucose control and sleep dysfunction which impair cognitive and psychomotor functions, and, in turn, increase driver risk. How this risk plays out in the real-world driving settings is terra incognita. Addressing this knowledge gap requires comprehensive observations of diabetes driver behavior and physiology in challenging settings where crashes are more likely to occur, such as stop-controlled traffic intersections, as in the current study of drivers with Type 1 Diabetes (T1DM). Methods: 32 active drivers from around Omaha, NE participated in 4-week, real-world study. Each participant's own vehicle was instrumented with an advanced telematics and camera system collecting driving sensor data and video. Videos were analyzed using computer vision models detecting traffic elements to identify stop signs. Stop sign detections and driver stopping trajectories were clustered to geolocate and extract driver-visited stop intersections. Driver videos were then annotated to record stopping behavior and key traffic characteristics. Stops were categorized as safe or unsafe based on traffic law. Results: Mixed effects logistic regression models examined how stopping behavior (safe vs. unsafe) in T1DM drivers was affected by 1) abnormal sleep, 2) obesity, and 3) poor glucose control. Model results indicate that one standard deviation increase in BMI (~7 points) in T1DM drivers associated with a 14.96 increase in unsafe stopping odds compared to similar controls. Abnormal sleep and glucose control were not associated with increased unsafe stopping. Conclusion: This study links chronic patterns of abnormal T1DM driver physiology, sleep, and health to driver safety risk at intersections, advancing models to identify real-world safety risk in diabetes drivers for clinical intervention and development of in-vehicle safety assistance technology., Comment: 23 pages, 7 figures, 10 tables

Recent efforts in interpretable deep learning models have shown that concept-based explanation methods achieve competitive accuracy with standard end-to-end models and enable reasoning and intervention about extracted high-level visual concepts from images, e.g., identifying the wing color and beak length for bird-species classification. However, these concept bottleneck models rely on a necessary and sufficient set of predefined concepts-which is intractable for complex tasks such as video classification. For complex tasks, the labels and the relationship between visual elements span many frames, e.g., identifying a bird flying or catching prey-necessitating concepts with various levels of abstraction. To this end, we present CoDEx, an automatic Concept Discovery and Extraction module that rigorously composes a necessary and sufficient set of concept abstractions for concept-based video classification. CoDEx identifies a rich set of complex concept abstractions from natural language explanations of videos-obviating the need to predefine the amorphous set of concepts. To demonstrate our method's viability, we construct two new public datasets that combine existing complex video classification datasets with short, crowd-sourced natural language explanations for their labels. Our method elicits inherent complex concept abstractions in natural language to generalize concept-bottleneck methods to complex tasks., Comment: 10 pages, Appendix: 2 pages

The aim of this study is to propose a modified Susceptible-Exposed-Infectious-Removed (SEIR) model that describes the behaviour of symptomatic, asymptomatic and hospitalized patients of COVID-19 epidemic, including the effect of demographic variation of population. It is shown that considering a population growth proportional to the total population leads to solutions with a qualitative behaviour different from the behaviour obtained in many studies, where constant growth ratio is assumed. An exhaustive theoretical study is carried out and the basic reproduction number $R_0$ is computed from the model equations. It is proved that if $R_0<1$ then the disease-free manifold is globally asymptotically stable, that is, the epidemics remits. Global and local stability of the equilibrium points is also studied. Numerical simulations are used to show the agreement between numerical results and theoretical properties. The model is fitted to experimental data corresponding to the pandemic evolution in the Rep\'ublica de Cuba, showing a proper behaviour of infected cases which let us think that can provide a correct estimation of asymptomatic cases. In conclusion, the model seems to be an adequate tool for the study and control of infectious diseases in particular the COVID-19 disease transmission.

Semantic code search is the task of retrieving a code snippet given a textual description of its functionality. Recent work has been focused on using similarity metrics between neural embeddings of text and code. However, current language models are known to struggle with longer, compositional text, and multi-step reasoning. To overcome this limitation, we propose supplementing the query sentence with a layout of its semantic structure. The semantic layout is used to break down the final reasoning decision into a series of lower-level decisions. We use a Neural Module Network architecture to implement this idea. We compare our model - NS3 (Neuro-Symbolic Semantic Search) - to a number of baselines, including state-of-the-art semantic code retrieval methods, and evaluate on two datasets - CodeSearchNet and Code Search and Question Answering. We demonstrate that our approach results in more precise code retrieval, and we study the effectiveness of our modular design when handling compositional queries.