Search

Considerable progress has recently been made in leveraging CLIP (Contrastive Language-Image Pre-Training) models for text-guided image manipulation. However, all existing works rely on additional generative models to ensure the quality of results, because CLIP alone cannot provide enough guidance information for fine-scale pixel-level changes. In this paper, we introduce CLIPVG, a text-guided image manipulation framework using differentiable vector graphics, which is also the first CLIP-based general image manipulation framework that does not require any additional generative models. We demonstrate that CLIPVG can not only achieve state-of-art performance in both semantic correctness and synthesis quality, but also is flexible enough to support various applications far beyond the capability of all existing methods., Comment: 8 pages, 10 figures, AAAI2023

Transparent objects are widely used in industrial automation and daily life. However, robust visual recognition and perception of transparent objects have always been a major challenge. Currently, most commercial-grade depth cameras are still not good at sensing the surfaces of transparent objects due to the refraction and reflection of light. In this work, we present a transformer-based transparent object depth estimation approach from a single RGB-D input. We observe that the global characteristics of the transformer make it easier to extract contextual information to perform depth estimation of transparent areas. In addition, to better enhance the fine-grained features, a feature fusion module (FFM) is designed to assist coherent prediction. Our empirical evidence demonstrates that our model delivers significant improvements in recent popular datasets, e.g., 25% gain on RMSE and 21% gain on REL compared to previous state-of-the-art convolutional-based counterparts in ClearGrasp dataset. Extensive results show that our transformer-based model enables better aggregation of the object's RGB and inaccurate depth information to obtain a better depth representation. Our code and the pre-trained model will be available at https://github.com/yuchendoudou/TODE., Comment: Submitted to ICRA2023

This work presents a next-generation human-robot interface that can infer and realize the user's manipulation intention via sight only. Specifically, we develop a system that integrates near-eye-tracking and robotic manipulation to enable user-specified actions (e.g., grasp, pick-and-place, etc), where visual information is merged with human attention to create a mapping for desired robot actions. To enable sight guided manipulation, a head-mounted near-eye-tracking device is developed to track the eyeball movements in real-time, so that the user's visual attention can be identified. To improve the grasping performance, a transformer based grasp model is then developed. Stacked transformer blocks are used to extract hierarchical features where the volumes of channels are expanded at each stage while squeezing the resolution of feature maps. Experimental validation demonstrates that the eye-tracking system yields low gaze estimation error and the grasping system yields promising results on multiple grasping datasets. This work is a proof of concept for gaze interaction-based assistive robot, which holds great promise to help the elder or upper limb disabilities in their daily lives. A demo video is available at https://www.youtube.com/watch?v=yuZ1hukYUrM, Comment: 6 pages

In this paper, we present the Multi-Forgery Detection Challenge held concurrently with the IEEE Computer Society Workshop on Biometrics at CVPR 2022. Our Multi-Forgery Detection Challenge aims to detect automatic image manipulations including but not limited to image editing, image synthesis, image generation, image photoshop, etc. Our challenge has attracted 674 teams from all over the world, with about 2000 valid result submission counts. We invited the Top 10 teams to present their solutions to the challenge, from which three teams are awarded prizes in the grand finale. In this paper, we present the solutions from the Top 3 teams, in order to boost the research work in the field of image forgery detection., Comment: Workshop and challenge summary paper, containing technical reports from different teams

Weibel-type instability can self-generate and amplify magnetic fields in both space and laboratory plasmas with temperature anisotropy. The electron Weibel instability has generally proven more challenging to measure than its ion counterpart owing to the much smaller inertia of electrons, resulting in a faster growth rate and smaller characteristic wavelength. Here, we have probed the evolution of the two-dimensional distribution of the magnetic field components and the current density due to electron Weibel instability, in $\rm CO_2$-ionized hydrogen gas (plasma) with picosecond resolution using a relativistic electron beam. We find that the wavenumber spectra of the magnetic fields are initially broad but eventually shrink to a narrow spectrum representing the dominant quasi-single mode. The measured $k$-resolved growth rates of the instability validate kinetic theory. Concurrently, self-organization of microscopic plasma currents is observed to amplify the current modulation magnitude that converts up to $\sim 1\%$ of the plasma thermal energy into magnetic energy., Comment: 24 pages, 4 figures

Generation and amplification of magnetic fields in plasmas is a long-standing topic that is of great interest to both plasma and space physics. The electron Weibel instability is a well-known mechanism responsible for self-generating magnetic fields in plasmas with temperature anisotropy and has been extensively investigated in both theory and simulations, yet experimental verification of this instability has been challenging. Recently, we demonstrated a new experimental platform that enables the controlled initialization of highly nonthermal and/or anisotropic plasma electron velocity distributions via optical-field ionization. Using an external electron probe bunch from a linear accelerator, the onset, saturation and decay of the self-generated magnetic fields due to electron Weibel instability were measured for the first time to our knowledge. In this paper, we will first present experimental results on time-resolved measurements of the Weibel magnetic fields in non-relativistic plasmas produced by Ti:Sapphire laser pulses (0.8 $\mu m$) and then discuss the feasibility of extending the study to quasi-relativistic regime by using intense $\rm CO_2$ (e.g., 9.2 $\mu m$) lasers to produce much hotter plasmas., Comment: 22 pages, 10 figures

Bacteria can spontaneously develop collective motions by aligning their motions in dense systems. Here, we show that bacteria can also respond collectively to an alternating electrical field and form dynamic clusters oscillating at the same frequency of the field. As the dynamic clusters go beyond a critical size, they split into smaller ones spontaneously. The critical size for splitting depends on the frequency of electric field and the concentration of bacteria. We show that instead of their biological activity, the physical properties of bacteria as charged particles are responsible for the formation of dynamic clusters. Electroconvective flows across the system play the key role in stabilizing the clusters. However, to form clusters, collective hydrodynamic cooperation between bacteria is important such that no aggregation occurs in dilute suspensions. The findings in this study illustrate that bio-systems can respond collectively to an external field, promising an effective way to control and modulate the behavior of organisms. Moreover, the controlled aggregation and condensation of bacteria offer a robust approach to improve the local concentration of bacteria for early and rapid detection, which has wide applications in clinics.