Search

This paper proposes the RePAIR dataset that represents a challenging benchmark to test modern computational and data driven methods for puzzle-solving and reassembly tasks. Our dataset has unique properties that are uncommon to current benchmarks for 2D and 3D puzzle solving. The fragments and fractures are realistic, caused by a collapse of a fresco during a World War II bombing at the Pompeii archaeological park. The fragments are also eroded and have missing pieces with irregular shapes and different dimensions, challenging further the reassembly algorithms. The dataset is multi-modal providing high resolution images with characteristic pictorial elements, detailed 3D scans of the fragments and meta-data annotated by the archaeologists. Ground truth has been generated through several years of unceasing fieldwork, including the excavation and cleaning of each fragment, followed by manual puzzle solving by archaeologists of a subset of approx. 1000 pieces among the 16000 available. After digitizing all the fragments in 3D, a benchmark was prepared to challenge current reassembly and puzzle-solving methods that often solve more simplistic synthetic scenarios. The tested baselines show that there clearly exists a gap to fill in solving this computationally complex problem., Comment: NeurIPS 2024, Track Datasets and Benchmarks, 10 pages

Jigsaw puzzle solving is a challenging task for computer vision since it requires high-level spatial and semantic reasoning. To solve the problem, existing approaches invariably use color and/or shape information but in many real-world scenarios, such as in archaeological fresco reconstruction, this kind of clues is often unreliable due to severe physical and pictorial deterioration of the individual fragments. This makes state-of-the-art approaches entirely unusable in practice. On the other hand, in such cases, simple geometrical patterns such as lines or curves offer a powerful yet unexplored clue. In an attempt to fill in this gap, in this paper we introduce a new challenging version of the puzzle solving problem in which one deliberately ignores conventional color and shape features and relies solely on the presence of linear geometrical patterns. The reconstruction process is then only driven by one of the most fundamental principles of Gestalt perceptual organization, namely Wertheimer's {\em law of good continuation}. In order to tackle this problem, we formulate the puzzle solving problem as the problem of finding a Nash equilibrium of a (noncooperative) multiplayer game and use classical multi-population replicator dynamics to solve it. The proposed approach is general and allows us to deal with pieces of arbitrary shape, size and orientation. We evaluate our approach on both synthetic and real-world data and compare it with state-of-the-art algorithms. The results show the intrinsic complexity of our purely line-based puzzle problem as well as the relative effectiveness of our game-theoretic formulation., Comment: to be published in ACCV2024

We discuss the open question of the relation between semantics and nominal class assignment in Swahili. We approach the problem from a computational perspective, aiming first to quantify the extent of this relation, and then to explicate its nature, taking extra care to suppress morphosyntactic confounds. Our results are the first of their kind, providing a quantitative evaluation of the semantic cohesion of each nominal class, as well as a nuanced taxonomic description of its semantic content., Comment: Tenth Italian Conference on Computational Linguistics (CliC-it-2024)

Maps of dynamics are effective representations of motion patterns learned from prior observations, with recent research demonstrating their ability to enhance performance in various downstream tasks such as human-aware robot navigation, long-term human motion prediction, and robot localization. Current advancements have primarily concentrated on methods for learning maps of human flow in environments where the flow is static, i.e., not assumed to change over time. In this paper we propose a method to update the CLiFF-map, one type of map of dynamics, for achieving efficient life-long robot operation. As new observations are collected, our goal is to update a CLiFF-map to effectively and accurately integrate new observations, while retaining relevant historic motion patterns. The proposed online update method maintains a probabilistic representation in each observed location, updating parameters by continuously tracking sufficient statistics. In experiments using both synthetic and real-world datasets, we show that our method is able to maintain accurate representations of human motion dynamics, contributing to high performance flow-compliant planning downstream tasks, while being orders of magnitude faster than the comparable baselines.

We propose an optimized optical vector network analyzer with automatic polarization control to stabilize the reference arm polarization throughout the sweep range. We demonstrate this technique, successfully removing the polarization-induced fading and measurement distortions in insertion loss by characterizing a 10 km multi-core fiber., Comment: Oral presentation at ECOC 2024 (Tu4A.4)- version with correction on the last sentence of paragraph 3 in the introduction

We prove that every quasi-elementary sub-Hopf algebra of the polynomial part of the odd primary Steenrod algebra must lie in a certain sub-Hopf algebra called $D$., Comment: 8 pages

Mult-layered meta-optics have enabled complex wavefront shaping beyond their single layer counterpart owing to the additional design variables afforded by each plane. For instance, complex amplitude modulation, generalized polarization transformations, and wide field of view are key attributes that fundamentally require multi-plane wavefront matching. Nevertheless, existing embodiments of bilayer metasurfaces have relied on configurations which suffer from Fresnel reflections, low mode confinement, or undesired resonances which compromise the intended response. Here, we introduce bilayer metasurfaces made of free-standing meta-atoms working in the visible spectrum. We demonstrate their use in wavefront shaping of linearly polarized light using pure geometric phase with diffraction efficiency of 80 % expanding previous literature on Pancharatnam-Berry phase metasurfaces which rely on circularly or elliptically polarized illumination. The fabrication relies on a two-step lithography and selective development processes which yield free standing, bilayer stacked metasurfaces, of 1200 nm total thickness. The metasurfaces comprise TiO2 nanofins with vertical side walls. Our work advances the nanofabrication of compound meta-optics and inspires new directions in wavefront shaping, metasurface integration, and polarization control., Comment: A. H. Dorrah, J. -S. Park, and A. Palmieri contributed equally to this work

Contrary to conventional massive MIMO cellular configurations plagued by inter-cell interference, cell-free massive MIMO systems distribute network resources across the coverage area, enabling users to connect with multiple access points (APs) and boosting both system capacity and fairness across user. In such systems, one critical functionality is the association between APs and users: determining the optimal association is indeed a combinatorial problem of prohibitive complexity. In this paper, a solution based on deep learning is thus proposed to solve the user clustering problem aimed at maximizing the sum spectral efficiency while controlling the number of active connections. The proposed solution can scale effectively with the number of users, leveraging long short-term memory cells to operate without the need for retraining. Numerical results show the effectiveness of the proposed solution, even in the presence of imperfect channel state information due to pilot contamination., Comment: Accepted to 25th IEEE International Workshop on Signal Processing Advances in Wireless Communications (SPAWC), 2024

Collaborative robots are increasingly popular for assisting humans at work and daily tasks. However, designing and setting up interfaces for human-robot collaboration is challenging, requiring the integration of multiple components, from perception and robot task control to the hardware itself. Frequently, this leads to highly customized solutions that rely on large amounts of costly training data, diverging from the ideal of flexible and general interfaces that empower robots to perceive and adapt to unstructured environments where they can naturally collaborate with humans. To overcome these challenges, this paper presents the Detection-Robot Management GPT (D-RMGPT), a robot-assisted assembly planner based on Large Multimodal Models (LMM). This system can assist inexperienced operators in assembly tasks without requiring any markers or previous training. D-RMGPT is composed of DetGPT-V and R-ManGPT. DetGPT-V, based on GPT-4V(vision), perceives the surrounding environment through one-shot analysis of prompted images of the current assembly stage and the list of components to be assembled. It identifies which components have already been assembled by analysing their features and assembly requirements. R-ManGPT, based on GPT-4, plans the next component to be assembled and generates the robot's discrete actions to deliver it to the human co-worker. Experimental tests on assembling a toy aircraft demonstrated that D-RMGPT is flexible and intuitive to use, achieving an assembly success rate of 83% while reducing the assembly time for inexperienced operators by 33% compared to the manual process. http://robotics-and-ai.github.io/LMMmodels/

We present ACME: A Chatbot for asylum-seeking Migrants in Europe. ACME relies on computational argumentation and aims to help migrants identify the highest level of protection they can apply for. This would contribute to a more sustainable migration by reducing the load on territorial commissions, Courts, and humanitarian organizations supporting asylum applicants. We describe the background context, system architecture, underlying technologies, and a case study used to validate the tool with domain experts., Comment: Accepted for publication at IEEE International Conference on Tools with Artificial Intelligence (ICTAI) @IEEE

In this paper, we prove a blow-up result for a generalized semilinear Euler-Poisson-Darboux equation with polynomially growing speed of propagation, when the power of the semilinear term is a shift of the Strauss' exponent for the classical semilinear wave equation. Our proof is based on a comparison argument of Kato-type for a second-order ODE with time-dependent coefficients, an integral representation formula by Yagdjian and the Radon transform. As byproduct of our method, we derive upper bound estimates for the lifespan which coincide with the sharp one for the classical semilinear wave equation in the critical case.

Predictive planning is a key capability for robots to efficiently and safely navigate populated environments. Particularly in densely crowded scenes, with uncertain human motion predictions, predictive path planning, and control can become expensive to compute in real time due to the curse of dimensionality. With the goal of achieving pro-active and legible robot motion in shared environments, in this paper we present HuMAN-MPC, a computationally efficient algorithm for Human Motion Aware Navigation using fast embedded Model Predictive Control. The approach consists of a novel model predictive control (MPC) formulation that leverages a fast state-of-the-art optimization backend based on a sequential quadratic programming real-time iteration scheme while also providing feasibility monitoring. Our experiments, in simulation and on a fully integrated ROS-based platform, show that the approach achieves great scalability with fast computation times without penalizing path quality and efficiency of the resulting avoidance behavior., Comment: published in Long-Term Human Motion Prediction (LHMP) Workshop at International Conference on Robotics and Automation (ICRA) 2024

Quantum compressed sensing is the fundamental tool for low-rank density matrix tomographic reconstruction in the informationally incomplete case. We examine situations where the acquired information is not enough to allow one to obtain a precise compressed sensing reconstruction. In this scenario, we propose a Deep Neural Network-based post-processing to improve the initial reconstruction provided by compressed sensing. The idea is to treat the estimated state as a noisy input for the network and perform a deep-supervised denoising task. After the network is applied, a projection onto the space of feasible density matrices is performed to obtain an improved final state estimation. We demonstrate through numerical experiments the improvement obtained by the denoising process and exploit the possibility of looping the inference scheme to obtain further advantages. Finally, we test the resilience of the approach to out-of-distribution data., Comment: 11 pages, 8 figures, github hyperlink included

Complex non-local behavior makes designing high efficiency and multifunctional metasurfaces a significant challenge. While using libraries of meta-atoms provide a simple and fast implementation methodology, pillar to pillar interaction often imposes performance limitations. On the other extreme, inverse design based on topology optimization leverages non-local coupling to achieve high efficiency, but leads to complex and difficult to fabricate structures. In this paper, we demonstrate numerically and experimentally a shape optimization method that enables high efficiency metasurfaces while providing direct control of the structure complexity. The proposed method provides a path towards manufacturability of inverse-designed high efficiency metasurfaces.

In the vibrant landscape of AI research, decentralised learning is gaining momentum. Decentralised learning allows individual nodes to keep data locally where they are generated and to share knowledge extracted from local data among themselves through an interactive process of collaborative refinement. This paradigm supports scenarios where data cannot leave local nodes due to privacy or sovereignty reasons or real-time constraints imposing proximity of models to locations where inference has to be carried out. The distributed nature of decentralised learning implies significant new research challenges with respect to centralised learning. Among them, in this paper, we focus on robustness issues. Specifically, we study the effect of nodes' disruption on the collective learning process. Assuming a given percentage of "central" nodes disappear from the network, we focus on different cases, characterised by (i) different distributions of data across nodes and (ii) different times when disruption occurs with respect to the start of the collaborative learning task. Through these configurations, we are able to show the non-trivial interplay between the properties of the network connecting nodes, the persistence of knowledge acquired collectively before disruption or lack thereof, and the effect of data availability pre- and post-disruption. Our results show that decentralised learning processes are remarkably robust to network disruption. As long as even minimum amounts of data remain available somewhere in the network, the learning process is able to recover from disruptions and achieve significant classification accuracy. This clearly varies depending on the remaining connectivity after disruption, but we show that even nodes that remain completely isolated can retain significant knowledge acquired before the disruption., Comment: Supported by the H2020 HumaneAI Net (952026), CHIST-ERA-19-XAI010 SAI, PNRR - M4C2 - Investimento 1.3, Partenariato Esteso PE00000013 FAIR, PNRR - M4C2 - Investimento 1.3, Partenariato Esteso PE00000001 RESTART

Remote direct memory access (RDMA) networks are being rapidly adopted into industry for their high speed, low latency, and reduced CPU overheads compared to traditional kernel-based TCP/IP networks. RDMA enables threads to access remote memory without interacting with another process. However, atomicity between local accesses and remote accesses is not guaranteed by the technology, hence complicating synchronization significantly. The current solution is to require threads wanting to access local memory in an RDMA-accessible region to pass through the RDMA card using a mechanism known as loopback, but this can quickly degrade performance. In this paper, we introduce ALock, a novel locking primitive designed for RDMA-based systems. ALock allows programmers to synchronize local and remote accesses without using loopback or remote procedure calls (RPCs). We draw inspiration from the classic Peterson's algorithm to create a hierarchical design that includes embedded MCS locks for two cohorts, remote and local. To evaluate the ALock we implement a distributed lock table, measuring throughput and latency in various cluster configurations and workloads. In workloads with a majority of local operations, the ALock outperforms competitors up to 29x and achieves a latency up to 20x faster., Comment: 13 pages, 6 figures, SPAA '24

The recent breakthroughs in the research on Large Language Models (LLMs) have triggered a transformation across several research domains. Notably, the integration of LLMs has greatly enhanced performance in robot Task And Motion Planning (TAMP). However, previous approaches often neglect the consideration of dynamic environments, i.e., the presence of dynamic objects such as humans. In this paper, we propose a novel approach to address this gap by incorporating human awareness into LLM-based robot task planning. To obtain an effective representation of the dynamic environment, our approach integrates humans' information into a hierarchical scene graph. To ensure the plan's executability, we leverage LLMs to ground the environmental topology and actionable knowledge into formal planning language. Most importantly, we use LLMs to predict future human activities and plan tasks for the robot considering the predictions. Our contribution facilitates the development of integrating human awareness into LLM-driven robot task planning, and paves the way for proactive robot decision-making in dynamic environments.

Recent advancements in Large Language Models (LLMs) have sparked a revolution across many research fields. In robotics, the integration of common-sense knowledge from LLMs into task and motion planning has drastically advanced the field by unlocking unprecedented levels of context awareness. Despite their vast collection of knowledge, large language models may generate infeasible plans due to hallucinations or missing domain information. To address these challenges and improve plan feasibility and computational efficiency, we introduce DELTA, a novel LLM-informed task planning approach. By using scene graphs as environment representations within LLMs, DELTA achieves rapid generation of precise planning problem descriptions. To enhance planning performance, DELTA decomposes long-term task goals with LLMs into an autoregressive sequence of sub-goals, enabling automated task planners to efficiently solve complex problems. In our extensive evaluation, we show that DELTA enables an efficient and fully automatic task planning pipeline, achieving higher planning success rates and significantly shorter planning times compared to the state of the art.

We present a new large dataset of indoor human and robot navigation and interaction, called TH\"OR-MAGNI, that is designed to facilitate research on social navigation: e.g., modelling and predicting human motion, analyzing goal-oriented interactions between humans and robots, and investigating visual attention in a social interaction context. TH\"OR-MAGNI was created to fill a gap in available datasets for human motion analysis and HRI. This gap is characterized by a lack of comprehensive inclusion of exogenous factors and essential target agent cues, which hinders the development of robust models capable of capturing the relationship between contextual cues and human behavior in different scenarios. Unlike existing datasets, TH\"OR-MAGNI includes a broader set of contextual features and offers multiple scenario variations to facilitate factor isolation. The dataset includes many social human-human and human-robot interaction scenarios, rich context annotations, and multi-modal data, such as walking trajectories, gaze tracking data, and lidar and camera streams recorded from a mobile robot. We also provide a set of tools for visualization and processing of the recorded data. TH\"OR-MAGNI is, to the best of our knowledge, unique in the amount and diversity of sensor data collected in a contextualized and socially dynamic environment, capturing natural human-robot interactions., Comment: Submitted to The International Journal of Robotics Research (IJRR) on 28 of February 2024

Fully decentralized learning is gaining momentum for training AI models at the Internet's edge, addressing infrastructure challenges and privacy concerns. In a decentralized machine learning system, data is distributed across multiple nodes, with each node training a local model based on its respective dataset. The local models are then shared and combined to form a global model capable of making accurate predictions on new data. Our exploration focuses on how different types of network structures influence the spreading of knowledge - the process by which nodes incorporate insights gained from learning patterns in data available on other nodes across the network. Specifically, this study investigates the intricate interplay between network structure and learning performance using three network topologies and six data distribution methods. These methods consider different vertex properties, including degree centrality, betweenness centrality, and clustering coefficient, along with whether nodes exhibit high or low values of these metrics. Our findings underscore the significance of global centrality metrics (degree, betweenness) in correlating with learning performance, while local clustering proves less predictive. We highlight the challenges in transferring knowledge from peripheral to central nodes, attributed to a dilution effect during model aggregation. Additionally, we observe that central nodes exert a pull effect, facilitating the spread of knowledge. In examining degree distribution, hubs in Barabasi-Albert networks positively impact learning for central nodes but exacerbate dilution when knowledge originates from peripheral nodes. Finally, we demonstrate the formidable challenge of knowledge circulation outside of segregated communities., Comment: Funding: H2020 HumaneAI Net (Grant N. 952026), CHIST-ERA SAI (CHIST-ERA-19-XAI010), PNRR FAIR (PE00000013), PNRR RESTART (PE00000001). arXiv admin note: text overlap with arXiv:2307.15947

The trajectory planning for a fleet of Automated Guided Vehicles (AGVs) on a roadmap is commonly referred to as the Multi-Agent Path Finding (MAPF) problem, the solution to which dictates each AGV's spatial and temporal location until it reaches it's goal without collision. When executing MAPF plans in dynamic workspaces, AGVs can be frequently delayed, e.g., due to encounters with humans or third-party vehicles. If the remainder of the AGVs keeps following their individual plans, synchrony of the fleet is lost and some AGVs may pass through roadmap intersections in a different order than originally planned. Although this could reduce the cumulative route completion time of the AGVs, generally, a change in the original ordering can cause conflicts such as deadlocks. In practice, synchrony is therefore often enforced by using a MAPF execution policy employing, e.g., an Action Dependency Graph (ADG) to maintain ordering. To safely re-order without introducing deadlocks, we present the concept of the Switchable Action Dependency Graph (SADG). Using the SADG, we formulate a comparatively low-dimensional Mixed-Integer Linear Program (MILP) that repeatedly re-orders AGVs in a recursively feasible manner, thus maintaining deadlock-free guarantees, while dynamically minimizing the cumulative route completion time of all AGVs. Various simulations validate the efficiency of our approach when compared to the original ADG method as well as robust MAPF solution approaches., Comment: IEEE Transactions on Robotics (T-Ro) preprint, 17 pages, 32 figures

This paper provides a perspective on the literature and current challenges in Multi-Agent Systems for interoperable robot navigation in industry. The focus is on the multi-agent decision stack for Autonomous Mobile Robots operating in mixed environments with humans, manually driven vehicles, and legacy Automated Guided Vehicles. We provide typical characteristics of such Multi-Agent Systems observed today and how these are expected to change on the short term due to the new standard VDA5050 and the interoperability framework OpenRMF. We present recent changes in fleet management standards and the role of open middleware frameworks like ROS2 reaching industrial-grade quality. Approaches to increase the robustness and performance of multi-robot navigation systems for transportation are discussed, and research opportunities are derived., Comment: 6 pages, 2 figures, presented in the Decision Making in Multi-Agent Systems Workshop at IROS2022

Vortices are ubiquitous in nature and can be observed in fluids, condensed matter, and even in the formation of galaxies. Light, too, can evolve like a vortex. Optical vortices are exploited in light-matter interaction, free-space communications, and imaging. Here, we introduce optical rotatum; a new degree-of-freedom of light in which an optical vortex experiences a quadratic chirp in its orbital angular momentum along the optical path. We show that such an adiabatic deformation of topology is associated with the accumulation of a Berry phase factor which in turn perturbs the propagation constant (spatial frequency) of the beam. Remarkably, the spatial structure of optical rotatum follows a logarithmic spiral; a signature that is commonly seen in the pattern formation of seashells and galaxies. Our work expands previous literature on structured light, offers new modalities for light-matter interaction, communications, and sensing, and hints to analogous effects in condensed matter physics and Bose-Einstein condensates., Comment: 24 Pages, 4 Main Figures, 2 Extended Figures

Fully decentralized learning enables the distribution of learning resources and decision-making capabilities across multiple user devices or nodes, and is rapidly gaining popularity due to its privacy-preserving and decentralized nature. Importantly, this crowdsourcing of the learning process allows the system to continue functioning even if some nodes are affected or disconnected. In a disaster scenario, communication infrastructure and centralized systems may be disrupted or completely unavailable, hindering the possibility of carrying out standard centralized learning tasks in these settings. Thus, fully decentralized learning can help in this case. However, transitioning from centralized to peer-to-peer communications introduces a dependency between the learning process and the topology of the communication graph among nodes. In a disaster scenario, even peer-to-peer communications are susceptible to abrupt changes, such as devices running out of battery or getting disconnected from others due to their position. In this study, we investigate the effects of various disruptions to peer-to-peer communications on decentralized learning in a disaster setting. We examine the resilience of a decentralized learning process when a subset of devices drop from the process abruptly. To this end, we analyze the difference between losing devices holding data, i.e., potential knowledge, vs. devices contributing only to the graph connectivity, i.e., with no data. Our findings on a Barabasi-Albert graph topology, where training data is distributed across nodes in an IID fashion, indicate that the accuracy of the learning process is more affected by a loss of connectivity than by a loss of data. Nevertheless, the network remains relatively robust, and the learning process can achieve a good level of accuracy., Comment: Accepted at IEEE ICT-DM 2023

Flat optics or metasurfaces have opened new frontiers in wavefront shaping and its applications. Polarization optics is one prominent area which has greatly benefited from the shape-birefringence of metasurfaces. However, flat optics comprising a single layer of meta-atoms can only perform a subset of polarization transformations, constrained by a symmetric Jones matrix. This limitation can be tackled using metasurfaces composed of bilayer meta-atoms but exhausting all possible combinations of geometries to build a bilayer metasurface library is a very daunting task. Consequently, bilayer metasurfaces have been widely treated as a cascade (product) of two decoupled single-layer metasurfaces. Here, we test the validity of this assumption by considering a metasurface made of TiO2 on fused silica substrate at a design wavelength of 532 nm. We explore regions in the design space where the coupling between the top and bottom layers can be neglected, i.e., producing a far-field response which approximates that of two decoupled single-layer metasurfaces. We complement this picture with the near-field analysis to explore the underlying physics in regions where both layers are strongly coupled. Our analysis is general and it allows the designer to efficiently build a multi-layer metasurface, either in transmission or reflection, by only running one full-wave simulation for a single-layer metasurface., Comment: 31 pages, 14 figures

Resource-efficient quantum state tomography is one of the key ingredients of future quantum technologies. In this work, we propose a new tomography protocol combining standard quantum state reconstruction methods with an attention-based neural network architecture. We show how the proposed protocol is able to improve the averaged fidelity reconstruction over linear inversion and maximum-likelihood estimation in the finite-statistics regime, reducing at least by an order of magnitude the amount of necessary training data. We demonstrate the potential use of our protocol in physically relevant scenarios, in particular, to certify metrological resources in the form of many-body entanglement generated during the spin squeezing protocols. This could be implemented with the current quantum simulator platforms, such as trapped ions, and ultra-cold atoms in optical lattices., Comment: The Author Accepted Manuscript (AAM)

Human motion prediction is important for mobile service robots and intelligent vehicles to operate safely and smoothly around people. The more accurate predictions are, particularly over extended periods of time, the better a system can, e.g., assess collision risks and plan ahead. In this paper, we propose to exploit maps of dynamics (MoDs, a class of general representations of place-dependent spatial motion patterns, learned from prior observations) for long-term human motion prediction (LHMP). We present a new MoD-informed human motion prediction approach, named CLiFF-LHMP, which is data efficient, explainable, and insensitive to errors from an upstream tracking system. Our approach uses CLiFF-map, a specific MoD trained with human motion data recorded in the same environment. We bias a constant velocity prediction with samples from the CLiFF-map to generate multi-modal trajectory predictions. In two public datasets we show that this algorithm outperforms the state of the art for predictions over very extended periods of time, achieving 45% more accurate prediction performance at 50s compared to the baseline., Comment: Accepted to the 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)