Search

Vision-Language Models (VLMs) have shown great success as foundational models for downstream vision and natural language applications in a variety of domains. However, these models lack the spatial understanding necessary for robotics applications where the agent must reason about the affordances provided by the 3D world around them. We present a system which trains on spatially-localized egocentric videos in order to connect visual input and task descriptions to predict a task's spatial affordance, that is the location where a person would go to accomplish the task. We show our approach outperforms the baseline of using a VLM to map similarity of a task's description over a set of location-tagged images. Our learning-based approach has less error both on predicting where a task may take place and on predicting what tasks are likely to happen at the current location. The resulting system enables robots to use egocentric sensing to navigate to physical locations of novel tasks specified in natural language.

Traditional methods for constructing high-quality, personalized head avatars from monocular videos demand extensive face captures and training time, posing a significant challenge for scalability. This paper introduces a novel approach to create high quality head avatar utilizing only a single or a few images per user. We learn a generative model for 3D animatable photo-realistic head avatar from a multi-view dataset of expressions from 2407 subjects, and leverage it as a prior for creating personalized avatar from few-shot images. Different from previous 3D-aware face generative models, our prior is built with a 3DMM-anchored neural radiance field backbone, which we show to be more effective for avatar creation through auto-decoding based on few-shot inputs. We also handle unstable 3DMM fitting by jointly optimizing the 3DMM fitting and camera calibration that leads to better few-shot adaptation. Our method demonstrates compelling results and outperforms existing state-of-the-art methods for few-shot avatar adaptation, paving the way for more efficient and personalized avatar creation.

We present the discovery of a peculiar dwarf nova KSP-OT-201712a using high-cadence, multi-color observations made with the Korea Microlensing Telescope Network. KSP-OT-201712a exhibits a rare presence of outbursts during standstills as well as strong H${\alpha}$ emission for a dwarf nova below the period minimum with an orbital period of 58.75 $\pm$ 0.02 minutes. The outburst cycles are ~ 6.6 days within standstills but increase to ~ 15 days outside of them. Both B-V and V-I colors become bluer and redder as the outburst luminosities increase and decrease, respectively, for the outburst within standstill, while they evolve in the opposite directions outside of the standstills. The presence of strong double-peaked H${\alpha}$ and weak He I emission lines with He/H flux ratio of 0.27, together with absorption lines of Mg b and Na D in the source, leads to the estimation Teff ~ 4570 $\pm$ 40 K, [Fe/H] ~ 0.06 $\pm$ 0.15 dex, and log g ~ 4.5 $\pm$ 0.1 for its secondary. KSP-OT-201712a is the second He-deficient dwarf nova below the period minimum, while the temperature of the secondary is measured for the first time in such objects. We identify it to be an ER UMa type dwarf nova suggesting that the evolution of dwarf novae across the period minimum is accompanied by large mass transfers. The high temperature of the secondary indicates that the system started its mass transfer when the secondary was about 93$\%$ of its main sequence age. The system will evolve to a helium cataclysmic variable or to AM CVn once its hydrogen envelope is exhausted before it explodes as a Type Ia supernova., Comment: 12 pages, 5 figures, accepted for publication in ApJ

We present Ego-Exo4D, a diverse, large-scale multimodal multiview video dataset and benchmark challenge. Ego-Exo4D centers around simultaneously-captured egocentric and exocentric video of skilled human activities (e.g., sports, music, dance, bike repair). 740 participants from 13 cities worldwide performed these activities in 123 different natural scene contexts, yielding long-form captures from 1 to 42 minutes each and 1,286 hours of video combined. The multimodal nature of the dataset is unprecedented: the video is accompanied by multichannel audio, eye gaze, 3D point clouds, camera poses, IMU, and multiple paired language descriptions -- including a novel "expert commentary" done by coaches and teachers and tailored to the skilled-activity domain. To push the frontier of first-person video understanding of skilled human activity, we also present a suite of benchmark tasks and their annotations, including fine-grained activity understanding, proficiency estimation, cross-view translation, and 3D hand/body pose. All resources are open sourced to fuel new research in the community. Project page: http://ego-exo4d-data.org/, Comment: updated baseline results and dataset statistics to match the released v2 data; added table to appendix comparing stats of Ego-Exo4D alongside other datasets

Registering clothes from 4D scans with vertex-accurate correspondence is challenging, yet important for dynamic appearance modeling and physics parameter estimation from real-world data. However, previous methods either rely on texture information, which is not always reliable, or achieve only coarse-level alignment. In this work, we present a novel approach to enabling accurate surface registration of texture-less clothes with large deformation. Our key idea is to effectively leverage a shape prior learned from pre-captured clothing using diffusion models. We also propose a multi-stage guidance scheme based on learned functional maps, which stabilizes registration for large-scale deformation even when they vary significantly from training data. Using high-fidelity real captured clothes, our experiments show that the proposed approach based on diffusion models generalizes better than surface registration with VAE or PCA-based priors, outperforming both optimization-based and learning-based non-rigid registration methods for both interpolation and extrapolation tests., Comment: Project page: https://www-users.cse.umn.edu/~guo00109/projects/3dv2024/

The understanding and quantification of ductility in crystalline metals, which has led to their widespread and effective usage as a structural material, is lacking in metallic glasses (MGs). Here, we introduce such a framework for ductility. This very practical framework is based on a MGs ability to support stable shear band growth, quantified in a stress gradient, gradSDB, which we measure and calculate for a range of MGs. Whether a MG behaves ductile or brittle in an application is determined by the comparison between gradsDB the applied stress field gradient, gradsapp. If gradsDB > gradsapp, the MG will behave brittle, if gradsDB < gradsapp, the MG will behave ductile, and gradsapp - gradsDB indicates how ductile. This framework can explain observed plastic properties of MGs and their apparent contradicting brittle and ductile characteristics. Looking forward, proposed framework provides the constitutive relation to quantitatively model their plastic behavior in any application, a requirement to use MGs as structural materials.

\object{SN 2021aefx} is a normal Type Ia Supernova (SN) with red excess emission over the first $\sim$ 2 days. We present detailed analysis of this SN using our high-cadence KMTNet multi-band photometry, spectroscopy, and publicly available data. We provide the first measurements of its epochs of explosion (MJD 59529.32 $\pm$ 0.16) as well as ``first light'' (MJD 59529.85 $\pm$ 0.55) associated with the main ejecta ${\rm{^{56}Ni}}$ distribution. This places our first detection of SN 2021aefx at $\sim -$0.5 hours since ``first light'', indicating the presence of additional power sources. Our peak-spectrum confirms its Type Ia sub-classification as intermediate between Core-Normal and Broad-Line, and we estimate the ejecta mass to be $\sim$ 1.34 $M_{\odot}$. The pre-peak spectral evolution identifies fast-expanding material reaching $>$ 40,000 km s$^{-1}$ (the fastest ever observed in Type Ia SNe) and at least two distinct homologously-expanding ejecta components: (1) a normal-velocity (12,400 km s$^{-1}$) component consistent with the typical photospheric evolution of Chandrasekhar-mass ejecta; and (2) a high-velocity (23,500 km s$^{-1}$) component visible during the first $\sim$ 3.6 days post-explosion, which locates the component within the outer $<$ 16\% of the ejecta mass. Asymmetric, subsonic explosion processes producing a non-spherical photosphere provide an explanation for the simultaneous presence of the two components, as well as the red excess emission via a slight ${\rm{^{56}Ni}}$ enrichment in the outer $\sim$ 0.5\% of the ejecta mass. Our spectrum from 300 days post-peak advances the constraint against non-degenerate companions and further supports a near-Chandrasekhar-mass explosion origin. Off-center ignited delayed-detonations of Chandrasekhar-mass white dwarfs may be responsible for the observed features of SN 2021aefx in some normal Type Ia SNe., Comment: Submitted for publication in ApJ. 29 pages, 14 figures, 4 tables

We present the discovery of 48 new and the analysis of 55, including 7 previously discovered, dwarf galaxy candidates around the giant spiral galaxy NGC~2997 using deep $BVI$ images from the KMTNet Supernova Program. Their $V$-band central surface brightness and total absolute magnitudes are in the range of 20.3--26.7 mag arcsec$^{-2}$ and --(8.02--17.69) mag, respectively, while the $I$-band effective radii are 0.14--2.97 kpc. We obtain $\alpha$ $\simeq$ --1.43 $\pm$ 0.02 for the faint-end slope of their luminosity function, comparable to previously measured values but shallower than theoretical predictions based on $\Lambda$CDM models. The distance-independent distributions of their mass and color suggest that the group could have recently accreted new massive members from the surrounding fields. The systematically bluer colors of the brighter members indicate younger stellar population and higher star formation activities in them, which appears to be consistent with similar findings from the SAGA or ELVES survey. We suggest that the massive and bluer dwarf galaxies in the group have experienced less environmental quenching due to their recent accretion, while environmental quenching is more effective for the low-mass members. The interpretation of NGC~2997 being populationally young with recent accretion of massive members is also consistent with the overall morphological distribution of the dwarf galaxies showing a lack of morphologically evolved candidates but a plethora of irregularly shaped ones. Our detection rate of dwarf galaxy candidates in the NGC~2997 group and their inferred star formation activities are comparable to those found in Milky Way analog systems from recent surveys within the magnitude limit M$_{V}$ $\lesssim$ --13 mag., Comment: 24 pages, 14 figures

In nuclear isotope and chemistry laboratories, hot cells and gloveboxes provide scientists with a controlled and safe environment to perform experiments. Working on experiments in these isolated containment cells requires scientists to be physically present. For hot cell work today, scientists manipulate equipment and radioactive material inside through a bilateral mechanical control mechanism. Motions produced outside the cell with the master control levers are mechanically transferred to the internal grippers inside the shielded containment cell. There is a growing need to have the capability to conduct experiments within these cells remotely. A simple method to enable remote manipulations within hot cell and glovebox cells is to mount two robotic arms inside a box to mimic the motions of human hands. An AR application was built in this work to allow a user wearing a Microsoft HoloLens 2 headset to teleoperate dual arm manipulators by grasping robotic end-effector digital replicas in AR from a remote location. In addition to the real-time replica of the physical robotic arms in AR, the application enables users to view a live video stream attached to the robotic arms and parse a 3D point cloud of 3D objects in their remote AR environment for better situational awareness. This work also provides users with virtual fixture to assist in manipulation and other teleoperation tasks., Comment: Abstract Submitted to the First International Workshop "Horizons of an Extended Robotics Reality" at the 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). https://sites.google.com/view/xr-robotics-iros2022/home?authuser=0

Clothes undergo complex geometric deformations, which lead to appearance changes. To edit human videos in a physically plausible way, a texture map must take into account not only the garment transformation induced by the body movements and clothes fitting, but also its 3D fine-grained surface geometry. This poses, however, a new challenge of 3D reconstruction of dynamic clothes from an image or a video. In this paper, we show that it is possible to edit dressed human images and videos without 3D reconstruction. We estimate a geometry aware texture map between the garment region in an image and the texture space, a.k.a, UV map. Our UV map is designed to preserve isometry with respect to the underlying 3D surface by making use of the 3D surface normals predicted from the image. Our approach captures the underlying geometry of the garment in a self-supervised way, requiring no ground truth annotation of UV maps and can be readily extended to predict temporally coherent UV maps. We demonstrate that our method outperforms the state-of-the-art human UV map estimation approaches on both real and synthetic data.