Search

Robot learning requires a considerable amount of high-quality data to realize the promise of generalization. However, large data sets are costly to collect in the real world. Physics simulators can cheaply generate vast data sets with broad coverage over states, actions, and environments. However, physics engines are fundamentally misspecified approximations to reality. This makes direct zero-shot transfer from simulation to reality challenging, especially in tasks where precise and force-sensitive manipulation is necessary. Thus, fine-tuning these policies with small real-world data sets is an appealing pathway for scaling robot learning. However, current reinforcement learning fine-tuning frameworks leverage general, unstructured exploration strategies which are too inefficient to make real-world adaptation practical. This paper introduces the Simulation-Guided Fine-tuning (SGFT) framework, which demonstrates how to extract structural priors from physics simulators to substantially accelerate real-world adaptation. Specifically, our approach uses a value function learned in simulation to guide real-world exploration. We demonstrate this approach across five real-world dexterous manipulation tasks where zero-shot sim-to-real transfer fails. We further demonstrate our framework substantially outperforms baseline fine-tuning methods, requiring up to an order of magnitude fewer real-world samples and succeeding at difficult tasks where prior approaches fail entirely. Last but not least, we provide theoretical justification for this new paradigm which underpins how SGFT can rapidly learn high-performance policies in the face of large sim-to-real dynamics gaps. Project webpage: https://weirdlabuw.github.io/sgft/{weirdlabuw.github.io/sgft}

We introduce exacting cardinals and a strengthening of these, ultraexacting cardinals. These are natural large cardinals defined equivalently as weak forms of rank-Berkeley cardinals, strong forms of J\'onsson cardinals, or in terms of principles of structural reflection. However, they challenge commonly held intuition on strong axioms of infinity. We prove that ultraexacting cardinals are consistent with Zermelo-Fraenkel Set Theory with the Axiom of Choice (ZFC) relative to the existence of an I0 embedding. However, the existence of an ultraexacting cardinal below a measurable cardinal implies the consistency of ZFC with a proper class of I0 embeddings, thus challenging the linear-incremental picture of the large cardinal hierarchy. We show that the existence of an exacting cardinal implies that V is not equal to HOD (G\"odel's universe of Hereditarily Ordinal Definable sets), showing that these cardinals surpass the current hierarchy of large cardinals consistent with ZFC. We prove that the consistency of ZFC with an exacting cardinal above an extendible cardinal refutes Woodin's HOD Conjecture and Ultimate-L Conjecture. Finally, we establish the consistency of ZFC with the existence of an exacting cardinal above an extendible cardinal from the consistency of ZF with certain large cardinals beyond choice., Comment: 39 pages

Integrating cameras into wireless smart rings has been challenging due to size and power constraints. We introduce IRIS, the first wireless vision-enabled smart ring system for smart home interactions. Equipped with a camera, Bluetooth radio, inertial measurement unit (IMU), and an onboard battery, IRIS meets the small size, weight, and power (SWaP) requirements for ring devices. IRIS is context-aware, adapting its gesture set to the detected device, and can last for 16-24 hours on a single charge. IRIS leverages the scene semantics to achieve instance-level device recognition. In a study involving 23 participants, IRIS consistently outpaced voice commands, with a higher proportion of participants expressing a preference for IRIS over voice commands regarding toggling a device's state, granular control, and social acceptability. Our work pushes the boundary of what is possible with ring form-factor devices, addressing system challenges and opening up novel interaction capabilities., Comment: 15 pages, 17 figures, 6 tables, to be published in UIST 2024

When an applicant files an online application, there is usually a requirement to fill the marks in the online form and also upload the marksheet in the portal for the verification. A system was built for reading the uploaded marksheet using OCR and automatically filling the rows/ columns in the online form. Though there are partial solutions to this problem - implemented using PyTesseract - the accuracy is low. Hence, the PaddleOCR was used to build the marksheet parser. Several pre-processing and post-processing steps were also performed. The system was tested and evaluated for seven states. Further work is being done and the system is being evaluated for more states and boards of India., Comment: 5 pages, 1 figure, 1 table

We deal with the complex issue of whether large cardinals are intrinsically justified principles of set theory (we call this the Intrinsicness Issue). In order to do this, we review, in a systematic fashion, (1.) the abstract principles that have been formulated to motivate them, as well as (2.) their mathematical expressions, and assess the justifiability of both on the grounds of the (iterative) concept of set. A parallel, but closely linked, issue is whether there exist mathematical principles able to yield all known large cardinals (we call this the Universality Issue), and we also test principles for their responses to this issue. Finally, we discuss the first author's Structural Reflection Principles (SRPs), and their response to Intrinsicness and Universality. We conclude the paper with some considerations on the global justifiability of SRPs, and on alternative construals of the concept of set also potentially able to intrinsically justify large cardinals.

We propose a new method for count-based exploration in high-dimensional state spaces. Unlike previous work which relies on density models, we show that counts can be derived by averaging samples from the Rademacher distribution (or coin flips). This insight is used to set up a simple supervised learning objective which, when optimized, yields a state's visitation count. We show that our method is significantly more effective at deducing ground-truth visitation counts than previous work; when used as an exploration bonus for a model-free reinforcement learning algorithm, it outperforms existing approaches on most of 9 challenging exploration tasks, including the Atari game Montezuma's Revenge., Comment: 11 pages (+9 appendix). Published as a conference paper at ICML 2023. Code available at https://github.com/samlobel/CFN/

One of the gnarliest challenges in reinforcement learning (RL) is exploration that scales to vast domains, where novelty-, or coverage-seeking behaviour falls short. Goal-directed, purposeful behaviours are able to overcome this, but rely on a good goal space. The core challenge in goal discovery is finding the right balance between generality (not hand-crafted) and tractability (useful, not too many). Our approach explicitly seeks the middle ground, enabling the human designer to specify a vast but meaningful proto-goal space, and an autonomous discovery process to refine this to a narrower space of controllable, reachable, novel, and relevant goals. The effectiveness of goal-conditioned exploration with the latter is then demonstrated in three challenging environments.

We unveil new patterns of Structural Reflection in the large-cardinal hierarchy below the first measurable cardinal. Namely, we give two different characterizations of strongly unfoldable and subtle cardinals in terms of a weak form of the principle of Structural Reflection, and also in terms of weak product structural reflection. Our analysis prompts the introduction of the new notion of $C^{(n)}$-strongly unfoldable cardinal for every natural number $n$, and we show that these cardinals form a natural hierarchy between strong unfoldable and subtle cardinals analogous to the known hierarchies of $C^{(n)}$-extendible and $\Sigma_n$-strong cardinals. These results show that the relatively low region of the large-cardinal hierarchy comprised between the first strongly unfoldable and the first subtle cardinals is completely analogous to the much higher region between the first strong and the first Woodin cardinals, and also to the much further upper region of the hierarchy ranging between the first supercompact and the first Vop\v{e}nka cardinals., Comment: 47 pages. Revised version

Face Recognition has proven to be one of the most successful technology and has impacted heterogeneous domains. Deep learning has proven to be the most successful at computer vision tasks because of its convolution-based architecture. Since the advent of deep learning, face recognition technology has had a substantial increase in its accuracy. In this paper, some of the most impactful face recognition systems were surveyed. Firstly, the paper gives an overview of a general face recognition system. Secondly, the survey covers various network architectures and training losses that have had a substantial impact. Finally, the paper talks about various databases that are used to evaluate the capabilities of a face recognition system.

We construct a model to explore the hydrodynamic interactions of active inclusions in curved biological membranes. The curved membrane is modelled as a two dimensional layer of highly viscous fluid, surrounded by external solvents of different viscosities. The active inclusions are modelled as point force dipoles. The point dipole limit is taken along a geodesic of the curved geometry, incorporating the change in orientation of the forces due to curvature. We demonstrate this explicitly for the case of a spherical membrane, leading to an analytic solution for the flow generated by a single inclusion. We further show that the flow field features an additional defect of negative index, arising from the membrane topology, which is not present in the planar version of the model. We finally explore the hydrodynamic interactions of a pair of inclusions in regimes of low and high curvature, as well as situations where the external fluid outside the membrane is confined. Our study suggests aggregation of dipoles in curved biological membranes of both low and high curvatures, under strong confinement. However, very high curvatures tend to destroy dipole aggregation, even under strong confinement., Comment: Julia Package available at https://github.com/sarthakbagaria/curved-membrane-fluid-dynamics

After discussing the limitations inherent to all set-theoretic reflection principles akin to those studied by A. L\'evy et. al. in the 1960's, we introduce new principles of reflection based on the general notion of \emph{Structural Reflection} and argue that they are in strong agreement with the conception of reflection implicit in Cantor's original idea of the unknowability of the \emph{Absolute}, which was subsequently developed in the works of Ackermann, L\'evy, G\"odel, Reinhardt, and others. We then present a comprehensive survey of results showing that different forms of the new principles of Structural Reflection are equivalent to well-known large cardinals axioms covering all regions of the large-cardinal hierarchy, thereby justifying the naturalness of the latter.

We study Structural Reflection beyond Vop\v{e}nka's Principle, at the level of almost-huge cardinals and higher, up to rank-into-rank embeddings. We identify and classify new large cardinal notions in that region that correspond to some form of what we call Exact Structural Reflection ($\mathrm{ESR}$). Namely, given cardinals $\kappa<\lambda$ and a class $\mathcal{C}$ of structures of the same type, the corresponding instance of $\mathrm{ESR}$ asserts that for every structure $A$ in $\mathcal{C}$ of rank $\lambda$, there is a structure $B$ in $\mathcal{C}$ of rank $\kappa$ and an elementary embedding of $B$ into $A$. Inspired by the statement of Chang's Conjecture, we also introduce and study sequential forms of $\mathrm{ESR}$, which, in the case of sequences of length $\omega$, turn out to be very strong. Indeed, when restricted to $\Pi_1$-definable classes of structures they follow from the existence of $I1$-embeddings, while for more complicated classes of structures, e.g., $\Sigma_2$, they are not known to be consistent. Thus, these principles unveil a new class of large cardinals that go beyond $I1$-embeddings, yet they may not fall into Kunen's Inconsistency., Comment: This is an update of the published version of the paper that corrects a problem in the definition of "weakly exact cardinals"