Search

Action recognition in dark, low-light (under-exposed) or noisy videos is a challenging task due to visibility degradation, which can hinder critical spatiotemporal details. This paper proposes MD-BERT, a novel multi-stream approach that integrates complementary pre-processing techniques such as gamma correction and histogram equalization alongside raw dark frames to address these challenges. We introduce the Dynamic Feature Fusion (DFF) module, extending existing attentional fusion methods to a three-stream setting, thereby capturing fine-grained and global contextual information across different brightness and contrast enhancements. The fused spatiotemporal features are then processed by a BERT-based temporal model, which leverages its bidirectional self-attention to effectively capture long-range dependencies and contextual relationships across frames. Extensive experiments on the ARID V1.0 and ARID V1.5 dark video datasets show that MD-BERT outperforms existing methods, establishing a new state-of-the-art performance. Ablation studies further highlight the individual contributions of each input stream and the effectiveness of the proposed DFF and BERT modules. The official website of this work is available at: https://github.com/HrishavBakulBarua/DarkBERT

Open optical networks have been considered to be important for cost-effectively building and operating the networks. Recently, the optical-circuit-switches (OCSes) have attracted industry and academia because of their cost efficiency and higher capacity than traditional electrical packet switches (EPSes) and reconfigurable optical add drop multiplexers (ROADMs). Though the open interfaces and control planes for traditional ROADMs and transponders have been defined by several standard-defining organizations (SDOs), those of OCSes have not. Considering that several OCSes have already been installed in production datacenter networks (DCNs) and several OCS products are on the market, bringing the openness and interoperability into the OCS-based networks has become important. Motivated by this fact, this paper investigates a software-defined networking (SDN) controller for open optical-circuit-switched networks. To this end, we identified the use cases of OCSes and derived the controller requirements for supporting them. We then proposed a multi-vendor (MV) OCS controller framework that satisfies the derived requirements; it was designed to quickly and consistently operate fiber paths upon receiving the operation requests. We validated our controller by implementing it and evaluating its performance on actual MV-OCS networks. It satisfied all the requirements, and fiber paths could be configured within 1.0 second by using our controller.

Many robot manipulation tasks require active or interactive exploration behavior in order to be performed successfully. Such tasks are ubiquitous in embodied domains, where agents must actively search for the information necessary for each stage of a task, e.g., moving the head of the robot to find information relevant to manipulation, or in multi-robot domains, where one scout robot may search for the information that another robot needs to make informed decisions. We identify these tasks with a new type of problem, factorized Contextual Markov Decision Processes, and propose DISaM, a dual-policy solution composed of an information-seeking policy that explores the environment to find the relevant contextual information and an information-receiving policy that exploits the context to achieve the manipulation goal. This factorization allows us to train both policies separately, using the information-receiving one to provide reward to train the information-seeking policy. At test time, the dual agent balances exploration and exploitation based on the uncertainty the manipulation policy has on what the next best action is. We demonstrate the capabilities of our dual policy solution in five manipulation tasks that require information-seeking behaviors, both in simulation and in the real-world, where DISaM significantly outperforms existing methods. More information at https://robin-lab.cs.utexas.edu/learning2look/., Comment: Project Website: https://robin-lab.cs.utexas.edu/learning2look/

While DNA origami is a powerful bottom-up fabrication technique, the physical and chemical stability of DNA nanostructures is generally limited to aqueous buffer conditions. Wet chemical silicification can stabilise these structures but does not add further functionality. Here, we demonstrate a versatile 3D nanofabrication technique to conformally coat micrometre-sized DNA origami crystals with functional metal oxides via atomic layer deposition (ALD). In addition to depositing homogenous and conformal nanometre-thin ZnO, TiO2, and IrO2 (multi)layers inside SiO2-stablised crystals, we establish a method to directly coat bare DNA crystals with ALD layers while maintaining the crystal integrity, enabled by critical point drying and low ALD process temperatures. As a proof-of-concept application, we demonstrate electrocatalytic water oxidation using ALD IrO2-coated DNA origami crystals, resulting in improved performance relative to planar films. Overall, our coating strategy establishes a tool set for designing custom-made 3D nanomaterials with precisely defined topologies and material compositions, combining the unique advantages of DNA origami and atomically controlled deposition of functional inorganic materials.

In online learning, the ability to provide quick and accurate feedback to learners is crucial. In skill-based learning, learners need to understand the underlying concepts and mechanisms of a skill to be able to apply it effectively. While videos are a common tool in online learning, they cannot comprehend or assess the skills being taught. Additionally, while Generative AI methods are effective in searching and retrieving answers from a text corpus, it remains unclear whether these methods exhibit any true understanding. This limits their ability to provide explanations of skills or help with problem-solving. This paper proposes a novel approach that merges Cognitive AI and Generative AI to address these challenges. We employ a structured knowledge representation, the TMK (Task-Method-Knowledge) model, to encode skills taught in an online Knowledge-based AI course. Leveraging techniques such as Large Language Models, Chain-of-Thought, and Iterative Refinement, we outline a framework for generating reasoned explanations in response to learners' questions about skills., Comment: 9 pages, 6 figures, 1 table

The Virtual Experimental Research Assistant (VERA) is an inquiry-based learning environment that empowers a learner to build conceptual models of complex ecological systems and experiment with agent-based simulations of the models. This study investigates the convergence of cognitive AI and generative AI for self-explanation in interactive AI agents such as VERA. From a cognitive AI viewpoint, we endow VERA with a functional model of its own design, knowledge, and reasoning represented in the Task--Method--Knowledge (TMK) language. From the perspective of generative AI, we use ChatGPT, LangChain, and Chain-of-Thought to answer user questions based on the VERA TMK model. Thus, we combine cognitive and generative AI to generate explanations about how VERA works and produces its answers. The preliminary evaluation of the generation of explanations in VERA on a bank of 66 questions derived from earlier work appears promising., Comment: 10 pages, 2 figures, 2 tables, 1 appendix, HEXED Workshop @EDM July 2024

The vision for 6G extends beyond mere communication, incorporating sensing capabilities to facilitate a diverse array of novel applications and services. However, the advent of joint communication and sensing (JCAS) technology introduces concerns regarding the handling of sensitive personally identifiable information (PII) pertaining to individuals and objects, along with external third-party data and disclosure. Consequently, JCAS-based applications are susceptible to privacy breaches, including location tracking, identity disclosure, profiling, and misuse of sensor data, raising significant implications under the European Union's general data protection regulation (GDPR) as well as other applicable standards. This paper critically examines emergent JCAS architectures and underscores the necessity for network functions to enable privacy-specific features in the 6G systems. We propose an enhanced JCAS architecture with additional network functions and interfaces, facilitating the management of sensing policies, consent information, and transparency guidelines, alongside the integration of sensing-specific functions and storage for sensing processing sessions. Furthermore, we conduct a comprehensive threat analysis for all interfaces, employing security threat model STRIDE and privacy threat model LINDDUN. We also summarise the identified threats using standard common weakness enumeration (CWE). Finally, we suggest the security and privacy controls as the mitigating strategies to counter the identified threats stemming from the JCAS architecture.

Mission critical communication (MCC) involves the exchange of information and data among emergency services, including the police, fire brigade, and other first responders, particularly during emergencies, disasters, or critical incidents. The widely-adopted TETRA (Terrestrial Trunked Radio)-based communication for mission critical services faces challenges including limited data capacity, coverage limitations, spectrum congestion, and security concerns. Therefore, as an alternative, mission critical communication over cellular networks (4G and 5G) has emerged. While cellular-based MCC enables features like real-time video streaming and high-speed data transmission, the involvement of network operators and application service providers in the MCC architecture raises privacy concerns for mission critical users and services. For instance, the disclosure of a policeman's location details to the network operator raises privacy concerns. To the best of our knowledge, no existing work considers the privacy issues in mission critical system with respect to 5G and upcoming technologies. Therefore, in this paper, we analyse the 3GPP standardised MCC architecture within the context of 5G core network concepts and assess the privacy implications for MC users, network entities, and MC servers. The privacy analysis adheres to the deployment strategies in the standard for MCC. Additionally, we explore emerging 6G technologies, such as off-network communications, joint communication and sensing, and non-3GPP communications, to identify privacy challenges in MCC architecture. Finally, we propose privacy controls to establish a next-generation privacy-preserving MCC architecture.

Human action or activity recognition in videos is a fundamental task in computer vision with applications in surveillance and monitoring, self-driving cars, sports analytics, human-robot interaction and many more. Traditional supervised methods require large annotated datasets for training, which are expensive and time-consuming to acquire. This work proposes a novel approach using Cross-Architecture Pseudo-Labeling with contrastive learning for semi-supervised action recognition. Our framework leverages both labeled and unlabelled data to robustly learn action representations in videos, combining pseudo-labeling with contrastive learning for effective learning from both types of samples. We introduce a novel cross-architecture approach where 3D Convolutional Neural Networks (3D CNNs) and video transformers (VIT) are utilised to capture different aspects of action representations; hence we call it ActNetFormer. The 3D CNNs excel at capturing spatial features and local dependencies in the temporal domain, while VIT excels at capturing long-range dependencies across frames. By integrating these complementary architectures within the ActNetFormer framework, our approach can effectively capture both local and global contextual information of an action. This comprehensive representation learning enables the model to achieve better performance in semi-supervised action recognition tasks by leveraging the strengths of each of these architectures. Experimental results on standard action recognition datasets demonstrate that our approach performs better than the existing methods, achieving state-of-the-art performance with only a fraction of labeled data. The official website of this work is available at: https://github.com/rana2149/ActNetFormer., Comment: Submitted for peer review

The creation of large, diverse, high-quality robot manipulation datasets is an important stepping stone on the path toward more capable and robust robotic manipulation policies. However, creating such datasets is challenging: collecting robot manipulation data in diverse environments poses logistical and safety challenges and requires substantial investments in hardware and human labour. As a result, even the most general robot manipulation policies today are mostly trained on data collected in a small number of environments with limited scene and task diversity. In this work, we introduce DROID (Distributed Robot Interaction Dataset), a diverse robot manipulation dataset with 76k demonstration trajectories or 350 hours of interaction data, collected across 564 scenes and 84 tasks by 50 data collectors in North America, Asia, and Europe over the course of 12 months. We demonstrate that training with DROID leads to policies with higher performance and improved generalization ability. We open source the full dataset, policy learning code, and a detailed guide for reproducing our robot hardware setup., Comment: Project website: https://droid-dataset.github.io/

A critical bottleneck limiting imitation learning in robotics is the lack of data. This problem is more severe in mobile manipulation, where collecting demonstrations is harder than in stationary manipulation due to the lack of available and easy-to-use teleoperation interfaces. In this work, we demonstrate TeleMoMa, a general and modular interface for whole-body teleoperation of mobile manipulators. TeleMoMa unifies multiple human interfaces including RGB and depth cameras, virtual reality controllers, keyboard, joysticks, etc., and any combination thereof. In its more accessible version, TeleMoMa works using simply vision (e.g., an RGB-D camera), lowering the entry bar for humans to provide mobile manipulation demonstrations. We demonstrate the versatility of TeleMoMa by teleoperating several existing mobile manipulators - PAL Tiago++, Toyota HSR, and Fetch - in simulation and the real world. We demonstrate the quality of the demonstrations collected with TeleMoMa by training imitation learning policies for mobile manipulation tasks involving synchronized whole-body motion. Finally, we also show that TeleMoMa's teleoperation channel enables teleoperation on site, looking at the robot, or remote, sending commands and observations through a computer network, and perform user studies to evaluate how easy it is for novice users to learn to collect demonstrations with different combinations of human interfaces enabled by our system. We hope TeleMoMa becomes a helpful tool for the community enabling researchers to collect whole-body mobile manipulation demonstrations. For more information and video results, https://robin-lab.cs.utexas.edu/telemoma-web.

This chapter explains the concept of \emph{Effective String Theories}(EST), and their success in explaining the results that Yang-Mills flux tubes behave, to a high degree of accuracy, like Bosonic Strings(BST). It describes EST's of L\"uscher and Weisz, and their principal conclusions. It then discusses the Polchinski-Strominger EST's. which are valid in all dimensions. It then describes the works by Drummond, and, the author and Peter Matlock, which extended the analysis to $R^{-3}$ order and showed, that even at that order the spectrum is that of BST. The chapter analyses the issues of string momentum in higher orders. It discusses at length the powerful covariant calculus, to systematically construct EST's to arbitrary orders. The most general actions to $R^{-7}$ order are shown to be governed by just two parameters. The works of Aharony and collaborators on the spectrum of EST's, both in static and conformal gauges,to $R^{-5}$ order, and their results that even at that order the ground state energy remains the same as that of BST, but the excited spectrum gets corrected for $D\,>\,3$, are explained. It discusses the simulation results for excited states. It also discusses the AdS-CFT approaches and thickness of flux tubes.Recent works by Ambjorn, Makeenko, and Sedrakyan on the path-integral approaches to this issue are also discussed and compared with the other approaches. It concludes with remarks on the significance of the results for QCD., Comment: This is a preprint of Chapter 22 of the following work: N.D. Hari Dass, Strings to Strings,2023,Springer reproduced with permission of Springer Nature Switzerland AG 2023. The final authenticated version is available online at the external DOI given below. In v2, typo in ref[47] has been corrected, and a new reference [48] has been added. Abstract has been modified in the light of these

Conceptual and simulation models can function as useful pedagogical tools, however it is important to categorize different outcomes when evaluating them in order to more meaningfully interpret results. VERA is a ecology-based conceptual modeling software that enables users to simulate interactions between biotics and abiotics in an ecosystem, allowing users to form and then verify hypothesis through observing a time series of the species populations. In this paper, we classify this time series into common patterns found in the domain of ecological modeling through two methods, hierarchical clustering and curve fitting, illustrating a general methodology for showing content validity when combining different pedagogical tools. When applied to a diverse sample of 263 models containing 971 time series collected from three different VERA user categories: a Georgia Tech (GATECH), North Georgia Technical College (NGTC), and ``Self Directed Learners'', results showed agreement between both classification methods on 89.38\% of the sample curves in the test set. This serves as a good indication that our methodology for determining content validity was successful., Comment: 16 pages, preprint

Energy efficiency in process industry and passive safety systems in nuclear power plants necessitate the use of buoyancy driven heat exchangers. The current study presents a 1-D numerical model of the buoyancy driven fluid system in a Coupled Natural Circulation Loop (CNCL) comprising of water as the working fluid. In this study water at different temperatures is considered as the operating fluid in each of the loops. The study employs a 1-D model derived using a semi-analytical Fourier series. A validation study was carried out using the relevant literature to verify the mathematical model. A detailed parametric study on individual NCLs and their coupled system was conducted by varying the cross-sectional area keeping the other parameters constant to gain insights into the effect of thermal coupling on the long-term transient dynamics of each of the individual loops of the CNCL system, for stable, neutrally stable, and unstable conditions. The dynamics were analysed using the difference between transient buoyancy and viscous forces, and it was found that the overall heat transfer coefficient influences the coupling behaviour and the dynamics of the component loops in a CNCL. At lower values of the overall heat transfer coefficient, the component loops in the CNCL nearly retain their independent behaviour, i.e., the component loops hardly influence each other. It was also found that the temperature dependent fluid properties influence the stability of the CNCL system in some cases.

Counterfeiting threatens human health, social equity, national security and global and local economies. Hardware-based cryptography that exploits physical unclonable functions (PUFs) provides the means for secure identification and authentication of products. While optical PUFs are among the hardest to replicate, they suffer from low encoding capacity and often complex and expensive read-out. Here we report PUF labels with nanoscale features and optical responses that arise from the guided self-assembly of plasmonic nanoparticles. Nanosphere lithography combined with DNA origami placement are used to create tightly packed randomised nanoparticle assemblies. Nanoscale variations within these assemblies define the scattering color of the individual spots that are arranged in a hexagonal lattice with spacing down to the optical resolution limit. Due to the nanoscale dimensions, the intrinsic randomness of the particle assemblies and their resulting optical responses, our PUFs are virtually impossible to replicate while they can be read-out with economical 3D-printed hardware.

Robot learning methods have recently made great strides, but generalization and robustness challenges still hinder their widespread deployment. Failing to detect and address potential failures renders state-of-the-art learning systems not combat-ready for high-stakes tasks. Recent advances in interactive imitation learning have presented a promising framework for human-robot teaming, enabling the robots to operate safely and continually improve their performances over long-term deployments. Nonetheless, existing methods typically require constant human supervision and preemptive feedback, limiting their practicality in realistic domains. This work aims to endow a robot with the ability to monitor and detect errors during task execution. We introduce a model-based runtime monitoring algorithm that learns from deployment data to detect system anomalies and anticipate failures. Unlike prior work that cannot foresee future failures or requires failure experiences for training, our method learns a latent-space dynamics model and a failure classifier, enabling our method to simulate future action outcomes and detect out-of-distribution and high-risk states preemptively. We train our method within an interactive imitation learning framework, where it continually updates the model from the experiences of the human-robot team collected using trustworthy deployments. Consequently, our method reduces the human workload needed over time while ensuring reliable task execution. Our method outperforms the baselines across system-level and unit-test metrics, with 23% and 40% higher success rates in simulation and on physical hardware, respectively. More information at https://ut-austin-rpl.github.io/sirius-runtime-monitor/

Boosting the task accuracy of tiny neural networks (TNNs) has become a fundamental challenge for enabling the deployments of TNNs on edge devices which are constrained by strict limitations in terms of memory, computation, bandwidth, and power supply. To this end, we propose a framework called NetDistiller to boost the achievable accuracy of TNNs by treating them as sub-networks of a weight-sharing teacher constructed by expanding the number of channels of the TNN. Specifically, the target TNN model is jointly trained with the weight-sharing teacher model via (1) gradient surgery to tackle the gradient conflicts between them and (2) uncertainty-aware distillation to mitigate the overfitting of the teacher model. Extensive experiments across diverse tasks validate NetDistiller's effectiveness in boosting TNNs' achievable accuracy over state-of-the-art methods. Our code is available at https://github.com/GATECH-EIC/NetDistiller.

Colloidal self-assembly allows rational design of structures on the micrometer and submicrometer scale. One architecture that can generate complete 3D photonic band gaps is the diamond cubic lattice, which has remained difficult to realize at length scales comparable to the wavelength of visible or ultraviolet light. Here, we demonstrate three-dimensional photonic crystals self-assembled from DNA origami that act as precisely programmable patchy colloids. Our DNA-based nanoscale tetrapods crystallize into a rod-connected diamond cubic lattice with a periodicity of 170 nm. This structure serves as a scaffold for atomic layer deposition of high refractive index materials such as TiO$_2$, yielding a tunable photonic band gap in the near-ultraviolet.

Large, high-capacity models trained on diverse datasets have shown remarkable successes on efficiently tackling downstream applications. In domains from NLP to Computer Vision, this has led to a consolidation of pretrained models, with general pretrained backbones serving as a starting point for many applications. Can such a consolidation happen in robotics? Conventionally, robotic learning methods train a separate model for every application, every robot, and even every environment. Can we instead train generalist X-robot policy that can be adapted efficiently to new robots, tasks, and environments? In this paper, we provide datasets in standardized data formats and models to make it possible to explore this possibility in the context of robotic manipulation, alongside experimental results that provide an example of effective X-robot policies. We assemble a dataset from 22 different robots collected through a collaboration between 21 institutions, demonstrating 527 skills (160266 tasks). We show that a high-capacity model trained on this data, which we call RT-X, exhibits positive transfer and improves the capabilities of multiple robots by leveraging experience from other platforms. More details can be found on the project website https://robotics-transformer-x.github.io., Comment: Project website: https://robotics-transformer-x.github.io