Search

Large language models (LLMs) have shown remarkable capabilities in various natural language tasks and are increasingly being applied in healthcare domains. This work demonstrates a new LLM-powered disease risk assessment approach via streaming human-AI conversation, eliminating the need for programming required by traditional machine learning approaches. In a COVID-19 severity risk assessment case study, we fine-tune pre-trained generative LLMs (e.g., Llama2-7b and Flan-t5-xl) using a few shots of natural language examples, comparing their performance with traditional classifiers (i.e., Logistic Regression, XGBoost, Random Forest) that are trained de novo using tabular data across various experimental settings. We develop a mobile application that uses these fine-tuned LLMs as its generative AI (GenAI) core to facilitate real-time interaction between clinicians and patients, providing no-code risk assessment through conversational interfaces. This integration not only allows for the use of streaming Questions and Answers (QA) as inputs but also offers personalized feature importance analysis derived from the LLM's attention layers, enhancing the interpretability of risk assessments. By achieving high Area Under the Curve (AUC) scores with a limited number of fine-tuning samples, our results demonstrate the potential of generative LLMs to outperform discriminative classification methods in low-data regimes, highlighting their real-world adaptability and effectiveness. This work aims to fill the existing gap in leveraging generative LLMs for interactive no-code risk assessment and to encourage further research in this emerging field.

The ocean ecology is badly impacted by large-scale oil spills, plastic waste, and chemical pollution, which destroy ecosystems and endanger marine life. Acknowledging the detrimental effects of oil spills on ecosystems, our research aims to establish the foundation for creative methods to lessen their impact. With an emphasis on the containment and prediction of oil spills, this research investigates the potential of acoustic levitation as a cutting-edge technique for environmental cleanup. Effectively separating and eliminating pollutants without causing additional ecological harm is a major issue for traditional oil spill cleanup techniques. Acoustic levitation provides a non-invasive, accurate, and effective alternative by using sound waves to precisely and subtly separate oil droplets from water in controlled environments. This proposed approach can reduce the negative effects on the environment and increase the efficacy of cleanup efforts. The findings have been examined and assessed by proof of concept experiments with oil droplets, identifying the relationship between the intensity of ultrasonic pressure and the proportion of oil droplets collected.

Recent increases in endpoint-based security events and threats compelled enterprise operations to switch to virtual desktop infrastructure and web-based applications. In addition to reducing potential hazards, this has guaranteed a consistent desktop environment for every user. On the other hand, the attack surface is greatly increased because all endpoints are connected to the company network, which could harbor malware and other advanced persistent threats. This results in a considerable loss of system resources on each individual endpoint. Hence our work proposes a standard called Colaboot that enables machines throughout a company to boot from a single operating system in order to address these problems and guarantee a consistent operating system environment that could be easily updated to the most recent security patches across all work stations.

Interconnected agents such as firms in a supply chain make simultaneous preparatory investments to increase chances of honouring their respective bilateral agreements. Failures cascade: if one fails their agreement, then so do all who follow in the chain. Thus, later agents' investments turn out to be pointless when there is an earlier failure. How losses are shared affects how agents invest to avoid the losses in the first place. In this way, a solution sets agent liabilities depending on the point of disruption and induces a supermodular investment game. We characterize all efficient solutions. These have the form that later agents -- who are not directly liable for the disruption -- still shoulder some of the losses, justified on the premise that they might have failed anyway. Importantly, we find that such indirect liabilities are necessary to avoid unbounded inefficiencies. Finally, we pinpoint one efficient solution with several desirable properties.

With challenges and limitations associated with security in the fintech industry, the rise to the need for data protection increases. However, the current existing passwordless and password-based peer to peer transactions in online banking systems are vulnerable to advanced forms of digital attacks. The influx of modern data protection methods keeps better records of the transactions, but it still does not address the issue of authentication and account takeovers during transactions. To the address the mentioned issue, this paper proposes a novel and robust peer to peer transaction system which employs best cloud security practices, proper use of cryptography and trusted computing to mitigate common vulnerabilities. We will be implementing FIDO2 compatible Smart Card to securely authenticate the user using physical smart cards and store the records in the cloud which enables access control by allowing access only when an access is requested. The standard incorporates multiple layers of security on cloud computing models to ensure secrecy of the said data. Services of the standard adhere to regulations provides by the government and assures privacy to the information of the payee or the end-user. The whole system has been implemented in the Internet of Things scenario.

Significant work has been done on advancing localization and mapping in underwater environments. Still, state-of-the-art methods are challenged by low-texture environments, which is common for underwater settings. This makes it difficult to use existing methods in diverse, real-world scenes. In this paper, we present TURTLMap, a novel solution that focuses on textureless underwater environments through a real-time localization and mapping method. We show that this method is low-cost, and capable of tracking the robot accurately, while constructing a dense map of a low-textured environment in real-time. We evaluate the proposed method using real-world data collected in an indoor water tank with a motion capture system and ground truth reference map. Qualitative and quantitative results validate the proposed system achieves accurate and robust localization and precise dense mapping, even when subject to wave conditions. The project page for TURTLMap is https://umfieldrobotics.github.io/TURTLMap., Comment: Accepted to IROS 2024

In an adaptive bitrate streaming application, the efficiency of video compression and the encoded video quality depend on both the video codec and the quality metric used to perform encoding optimization. The development of such a quality metric need large scale subjective datasets. In this work we merge several datasets into one to support the creation of a metric tailored for video compression and scaling. We proposed a set of HEVC lightweight features to boost performance of the metrics. Our metrics can be computed from tightly coupled encoding process with 4% compute overhead or from the decoding process in real-time. The proposed method can achieve better correlation than VMAF and P.1204.3. It can extrapolate to different dynamic ranges, and is suitable for real-time video quality metrics delivery in the bitstream. The performance is verified by in-distribution and cross-dataset tests. This work paves the way for adaptive client-side heuristics, real-time segment optimization, dynamic bitrate capping, and quality-dependent post-processing neural network switching, etc., Comment: Accepted at Picture Coding Symposium 2024

Preference-aligned robot navigation in human environments is typically achieved through learning-based approaches, utilizing user feedback or demonstrations for personalization. However, personal preferences are subject to change and might even be context-dependent. Yet traditional reinforcement learning (RL) approaches with static reward functions often fall short in adapting to these varying user preferences, inevitably reflecting demonstrations once training is completed. This paper introduces a framework that combines multi-objective reinforcement learning (MORL) with demonstration-based learning. Our approach allows for dynamic adaptation to changing user preferences without retraining. It fluently modulates between reward-defined preference objectives and the amount of demonstration data reflection. Through rigorous evaluations, including a sim-to-real transfer on two robots, we demonstrate our framework's capability to reflect user preferences accurately while achieving high navigational performance in terms of collision avoidance and goal pursuance.

Causal learning is a fundamental problem in statistics and science, offering insights into predicting the effects of unseen treatments on a system. Despite recent advances in this topic, most existing causal discovery algorithms operate under two key assumptions: (i) the underlying graph is acyclic, and (ii) the available data is complete. These assumptions can be problematic as many real-world systems contain feedback loops (e.g., biological systems), and practical scenarios frequently involve missing data. In this work, we propose a novel framework, named MissNODAGS, for learning cyclic causal graphs from partially missing data. Under the additive noise model, MissNODAGS learns the causal graph by alternating between imputing the missing data and maximizing the expected log-likelihood of the visible part of the data in each training step, following the principles of the expectation-maximization (EM) framework. Through synthetic experiments and real-world single-cell perturbation data, we demonstrate improved performance when compared to using state-of-the-art imputation techniques followed by causal learning on partially missing interventional data.

A standardized control system called Metaverse Extended Reality Portal (MERP) is presented as a solution to the issues with conventional VR eyewear. The MERP system improves user awareness of the physical world while offering an immersive 3D view of the metaverse by using a shouldermounted projector to display a Heads-Up Display (HUD) in a designated Metaverse Experience Room. To provide natural and secure interaction inside the metaverse, a compass module and gyroscope integration enable accurate mapping of real-world motions to avatar actions. Through user tests and research, the MERP system shows that it may reduce mishaps brought on by poor spatial awareness, offering an improved metaverse experience and laying the groundwork for future developments in virtual reality technology. MERP, which is compared with existing Virtual Reality (VR) glasses used to traverse the metaverse, is projected to become a seamless, novel and better alternative. Existing VR headsets and AR glasses have well-known drawbacks that making them ineffective for prolonged usage as it causes harm to the eyes.

We investigate whether region-based representations are effective for recognition. Regions were once a mainstay in recognition approaches, but pixel and patch-based features are now used almost exclusively. We show that recent class-agnostic segmenters like SAM can be effectively combined with strong unsupervised representations like DINOv2 and used for a wide variety of tasks, including semantic segmentation, object-based image retrieval, and multi-image analysis. Once the masks and features are extracted, these representations, even with linear decoders, enable competitive performance, making them well suited to applications that require custom queries. The compactness of the representation also makes it well-suited to video analysis and other problems requiring inference across many images., Comment: CVPR 2024 Camera Ready; website: https://regionreps.web.illinois.edu/

Autonomous underwater vehicles often perform surveys that capture multiple views of targets in order to provide more information for human operators or automatic target recognition algorithms. In this work, we address the problem of choosing the most informative views that minimize survey time while maximizing classifier accuracy. We introduce a novel active perception framework for multi-view adaptive surveying and reacquisition using side scan sonar imagery. Our framework addresses this challenge by using a graph formulation for the adaptive survey task. We then use Graph Neural Networks (GNNs) to both classify acquired sonar views and to choose the next best view based on the collected data. We evaluate our method using simulated surveys in a high-fidelity side scan sonar simulator. Our results demonstrate that our approach is able to surpass the state-of-the-art in classification accuracy and survey efficiency. This framework is a promising approach for more efficient autonomous missions involving side scan sonar, such as underwater exploration, marine archaeology, and environmental monitoring.

In the `Many Interacting Worlds' (MIW) discrete Hamiltonian system approximation of Schr\"odinger's wave equation, introduced in \cite{hall_2014}, convergence of ground states to the Normal ground state of the quantum harmonic oscillator, via Stein's method, in Wasserstein-$1$ distance with rate $\mathcal{O}(\sqrt{\log N}/N)$ has been shown in McKeague-Levin (2016), Chen-Thanh (2023), McKeague-Swan (2023). In this context, we construct approximate higher energy states of the MIW system, and show their convergence with the same rate in Wasserstein-$1$ distance to higher energy states of the quantum harmonic oscillator. In terms of techniques, we apply the `differential equation' approach to Stein's method, which allows to handle behavior near zeros of the higher energy states., Comment: 32 pages, 2 figures

Open-source benchmark datasets have been a critical component for advancing machine learning for robot perception in terrestrial applications. Benchmark datasets enable the widespread development of state-of-the-art machine learning methods, which require large datasets for training, validation, and thorough comparison to competing approaches. Underwater environments impose several operational challenges that hinder efforts to collect large benchmark datasets for marine robot perception. Furthermore, a low abundance of targets of interest relative to the size of the search space leads to increased time and cost required to collect useful datasets for a specific task. As a result, there is limited availability of labeled benchmark datasets for underwater applications. We present the AI4Shipwrecks dataset, which consists of 28 distinct shipwrecks totaling 286 high-resolution labeled side scan sonar images to advance the state-of-the-art in autonomous sonar image understanding. We leverage the unique abundance of targets in Thunder Bay National Marine Sanctuary in Lake Huron, MI, to collect and compile a sonar imagery benchmark dataset through surveys with an autonomous underwater vehicle (AUV). We consulted with expert marine archaeologists for the labeling of robotically gathered data. We then leverage this dataset to perform benchmark experiments for comparison of state-of-the-art supervised segmentation methods, and we present insights on opportunities and open challenges for the field. The dataset and benchmarking tools will be released as an open-source benchmark dataset to spur innovation in machine learning for Great Lakes and ocean exploration. The dataset and accompanying software are available at https://umfieldrobotics.github.io/ai4shipwrecks/., Comment: Project website link: https://umfieldrobotics.github.io/ai4shipwrecks/

Amazon Locker is a self-service delivery or pickup location where customers can pick up packages and drop off returns. A basic first-come-first-served policy for accepting package delivery requests to lockers results in lockers becoming full with standard shipping speed (3-5 day shipping) packages, and leaving no space left for expedited packages which are mostly Next-Day or Two-Day shipping. This paper proposes a solution to the problem of determining how much locker capacity to reserve for different ship-option packages. Yield management is a much researched field with popular applications in the airline, car rental, and hotel industries. However, Amazon Locker poses a unique challenge in this field since the number of days a package will wait in a locker (package dwell time) is, in general, unknown. The proposed solution combines machine learning techniques to predict locker demand and package dwell time, and linear programming to maximize throughput in lockers. The decision variables from this optimization provide optimal capacity reservation values for different ship options. This resulted in a year-over-year increase of 9% in Locker throughput worldwide during holiday season of 2018, impacting millions of customers., Comment: 22 pages, 10 figues

Financial markets are undergoing an unprecedented transformation. Technological advances have brought major improvements to the operations of financial services. While these advances promote improved accessibility and convenience, traditional finance shortcomings like lack of transparency and moral hazard frictions continue to plague centralized platforms, imposing societal costs. In this paper, we argue how these shortcomings and frictions are being mitigated by the decentralized finance (DeFi) ecosystem. We delve into the workings of smart contracts, the backbone of DeFi transactions, with an emphasis on those underpinning token exchange and lending services. We highlight the pros and cons of the novel form of decentralized governance introduced via the ownership of governance tokens. Despite its potential, the current DeFi infrastructure introduces operational risks to users, which we segment into five primary categories: consensus mechanisms, protocol, oracle, frontrunning, and systemic risks. We conclude by emphasizing the need for future research to focus on the scalability of existing blockchains, the improved design and interoperability of DeFi protocols, and the rigorous auditing of smart contracts., Comment: 2

Consider the asymmetric nearest-neighbor exclusion process (ASEP) on ${\mathbb Z}$ with single particle drift $\gamma>0$, starting from a Bernoulli product invariant measure $\nu_\rho$ with density $\rho$. It is known that the position $X_{N}$ of a tagged particle, say initially at the origin, at time $N$ satisfies an a.s. law of large numbers $\frac{1}{N}X_N \rightarrow \gamma(1-\rho)$ as $N\uparrow\infty$. In this context, we study the `typical' behavior of the tagged particle and `bulk' density evolution subject to `atypical' events $\{X_N\geq AN\}$ or $\{X_N\leq AN\}$ for $A\neq \gamma(1-\rho)$. We detail different structures, depending on whether $A<0$, $0\leq A< \gamma(1-\rho)$, $\gamma(1-\rho)

Foresighted robot navigation in dynamic indoor environments with cost-efficient hardware necessitates the use of a lightweight yet dependable controller. So inferring the scene dynamics from sensor readings without explicit object tracking is a pivotal aspect of foresighted navigation among pedestrians. In this paper, we introduce a spatiotemporal attention pipeline for enhanced navigation based on 2D~lidar sensor readings. This pipeline is complemented by a novel lidar-state representation that emphasizes dynamic obstacles over static ones. Subsequently, the attention mechanism enables selective scene perception across both space and time, resulting in improved overall navigation performance within dynamic scenarios. We thoroughly evaluated the approach in different scenarios and simulators, finding excellent generalization to unseen environments. The results demonstrate outstanding performance compared to state-of-the-art methods, thereby enabling the seamless deployment of the learned controller on a real robot.

The paper investigates using a Large Language Model (LLM) to automatically perform web software tasks using click, scroll, and text input operations. Previous approaches, such as reinforcement learning (RL) or imitation learning, are inefficient to train and task-specific. Our method uses filtered Document Object Model (DOM) elements as observations and performs tasks step-by-step, sequentially generating small programs based on the current observations. We use in-context learning, either benefiting from a single manually provided example, or an automatically generated example based on a successful zero-shot trial. We evaluate the proposed method on the MiniWob++ benchmark. With only one in-context example, our WebWISE method achieves similar or better performance than other methods that require many demonstrations or trials.

In this paper, we address the problem of sim-to-real transfer for object segmentation when there is no access to real examples of an object of interest during training, i.e. zero-shot sim-to-real transfer for segmentation. We focus on the application of shipwreck segmentation in side scan sonar imagery. Our novel segmentation network, STARS, addresses this challenge by fusing a predicted deformation field and anomaly volume, allowing it to generalize better to real sonar images and achieve more effective zero-shot sim-to-real transfer for image segmentation. We evaluate the sim-to-real transfer capabilities of our method on a real, expert-labeled side scan sonar dataset of shipwrecks collected from field work surveys with an autonomous underwater vehicle (AUV). STARS is trained entirely in simulation and performs zero-shot shipwreck segmentation with no additional fine-tuning on real data. Our method provides a significant 20% increase in segmentation performance for the targeted shipwreck class compared to the best baseline.

Passwordless authentication was first tested for seamless and secure merchant payments without the use of passwords or pins. It opened a whole new world of authentications giving up the former reliance on traditional passwords. It relied on the W3C Web Authentication (WebAuthn) and Client to Authenticator Protocol (CTAP) standards to use the public key cryptosystem to uniquely attest a user's device and then their identity. These standards comprise of the FIDO authentication standard. As the popularity of passwordless is increasing, more and more users and service providers are adopting to it. However, the concept of device attestation makes it device-specific for a user. It makes it difficult for a user to switch devices. FIDO Passkeys were aimed at solving the same, synchronizing the private cryptographic keys across multiple devices so that the user can perform passwordless authentication even from devices not explicitly enrolled with the service provider. However, passkeys have certain drawbacks including that it uses proprietary end to end encryption algorithms, all keys pass through proprietary cloud provider, and it is usually not very seamless when dealing with cross-platform key synchronization. To deal with the problems and drawbacks of FIDO Passkeys, the paper proposes a novel private key management system for passwordless authentication called Transferable User Secret on Hardware Key (TUSH-Key). TUSH-Key allows cross-platform synchronization of devices for seamless passwordless logins with FIDO2 specifications.

Autonomous vehicles and Advanced Driving Assistance Systems (ADAS) have the potential to radically change the way we travel. Many such vehicles currently rely on segmentation and object detection algorithms to detect and track objects around its surrounding. The data collected from the vehicles are often sent to cloud servers to facilitate continual/life-long learning of these algorithms. Considering the bandwidth constraints, the data is compressed before sending it to servers, where it is typically decompressed for training and analysis. In this work, we propose the use of a learning-based compression Codec to reduce the overhead in latency incurred for the decompression operation in the standard pipeline. We demonstrate that the learned compressed representation can also be used to perform tasks like semantic segmentation in addition to decompression to obtain the images. We experimentally validate the proposed pipeline on the Cityscapes dataset, where we achieve a compression factor up to $66 \times$ while preserving the information required to perform segmentation with a dice coefficient of $0.84$ as compared to $0.88$ achieved using decompressed images while reducing the overall compute by $11\%$., Comment: Accepted at ICME 2023 (Industry Track)